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Foreword 



When Australian students leave school, our community expects that they are 
confident and productive users of information and communication technologies 
(iCT) and understand the impact of ICT on society. 

This ICT Literacy Report represents the findings from the first national 
assessment of the ICT literacy of Australian school students. The ICT Literacy 
report is the third published as part of the National Assessment Program, and 
follows the 2003 National Year 6 Science Report and the Civics and Citizenship 
Years 6 & 10 Report 2004. The next national iCT assessment is due in 2008. 

The assessment, conducted towards the end of 2005, established a single 
ICT literacy scale against which the achievements of students at Years 6 and 
10 can be reported and proficiency levels linked to descriptions of student 
performance. 

The report compares results by State and Territory, it also reports on 
achievement according to gender, socio-economic group, indigenous status, 
language background and geographical location. 

The commitment of principals, teachers and students at government. Catholic 
and independent schools around Australia through their participation in the 
2005 assessment has provided a valuable evidence base of ICT literacy in 
Australian schools. 

Particular thanks are due to the people responsible for developing and 
administering the assessments on behalf of MCEETYA - the Performance 
Measurement and Reporting Taskforce and its Benchmarking and Educational 
Measurement Unit. 



This report will be of vital interest to everyone -teachers, educators, employers, 
parents and others - with a stake in how our young people are being equipped 
to use and understand ICT technology in the new century. 



Rachel Hunter 

Chair 

Performance Measurement and Reporting Taskforce 
July 2007 
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Executive Summary 



Australia’s national goals for schooling assert that when students leave school 
they should be: confident, creative and productive users of new technologies, 
particularly information and communication technologies, and understand 
the impact of those technologies on society (MCEETYA, 1999: Goal 1.6). The 
Australian National Assessment Program includes the systematic assessment 
of the extent to which this goal is being achieved through triennial sample 
surveys of students in Years 6 and 10. 

This report is based on the assessment of iCT literacy conducted in October 
2005. it describes the development of a computer-based tool for assessing 
ICT literacy among school students and the application of that tool with a 
nationally representative sample of approximately 7,400 students from Years 6 
and 1 0 in nearly 520 Australian schools. The report describes the development, 
validation and refinement of a progress map that identifies a progression of 
ICT literacy, it describes the ICT literacy levels of Australian school students 
overall and for particular groups of students. 



Definition of ICT Literacy 

For the purpose of this assessment iCT literacy is defined as: 

the ability of individuals to use ICT appropriately to access, manage, 
integrate and evaluate information, develop new understandings, and 
communicate with others in order to participate effectively in society 
(MCEETYA, 2005). 



From the definition a conception of student progress in ICT was formulated 
in terms of three “strands”: working with information; creating and sharing 
information; and using ICT responsibly. 



Assessment Method 

The assessment instrument was computer-based and was administered using 
sets of networked laptop computers delivered to each sampled school. The 
assessment instrument included simulated ICT screens that behaved the same 
as common application programs and authentic applications of real software 
to larger tasks so as to produce work for subsequent assessment. Requiring 
students to complete tasks in authentic contexts was fundamental to the 
design of the Australian national ICT Literacy assessment. 

The assessment instrument consisted of seven discrete thematic modules. 
One module, the General Skills Test, included only simulation and multiple- 
choice assessment items. Six of the modules, the Hybrid Assessment Modules, 
contained conventional simulation, multiple-choice and constructed response 
items with live application software. All students first completed the General 
Skills Test and then two Hybrid Assessment Modules. This ensured that the 
assessment instrument accessed what was common to ICT literacy across a 
range of authentic contexts. 

The assessment was completed by 3,746 Year 6 and 3,647 Year 10 students 
from 264 primary and 253 secondary schools across Australia. The sample 
was selected by cluster sampling methods to ensure that the results accurately 
represented the Australian population of Year 6 and Year 10 students. 



ICT Literacy Scale 

Item response modelling (the Rasch model) was used to analyse the pattern 
of student responses (which items and how many items they successfully 
completed). This process was the key to generating a single scale on which 
the items from each of the different assessment modules could be located and 
against which the students’ achievement could be reported. A large number 
of students completed each of the modules and the rotational design ensured 
that every possible combination was adequately covered. The analyses also 
showed that the items formed one dimension and that the scale was reliable in 
the sense of being internally consistent (the person separation index was 0.93 
on a possible range from zero to one). 

The ICT literacy scale was fixed so that the mean score for Year 6 students was 
400 and the standard deviation for Year 6 students was 100 points. The choice 
of these parameters meant that about two-thirds of the Year 6 students would 
have ICT literacy scores between 300 and 500 points. It follows from setting 
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these scale points that for Year 10 the mean ICT literacy score was 550 and the 
standard deviation was 97.5. in other words there was a clear difference in the 
ICT literacy of students in Year 6 and Year 10. 



ICT Literacy Profile 



The items distributed across the iCT literacy scale were used to develop a 
progress map that could be interpreted in terms of the skills and understandings 
demonstrated by students in their responses to the items, in this case six 
proficiency levels were defined and descriptions were developed to characterise 
typical student performance at each level. The profile is shown in Table ESI 
with the percentage of students in each proficiency level. The levels and the 
percentage in each level are used to summarise the performance of students 
overall, to compare performances across subgroups of students. These data 
are represented in Figure ESI. 

ICT Literacy Proficiency Distribution 



Year 6 Year 10 

Proficient Proficient 

Standard Standard 
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Level 1 Level 2 Level 3 Level 4 Level 5+ 



Proficiency Level 



Figure ESI: Distribution of Year 6 and Year 10 Students over ICT Proficiency Levels 
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Table ESI: ICT Literacy Profiles for Year 6 and Year 10 



Year 6 



Year 10 



0% 


Level 6 

Students working at level 6 create information products that show evidence 
of technical proficiency, and careful planning and review. They use software 
features to organise information and to synthesise and represent data as 
integrated complete information products. They design information products 
consistent with the conventions of specific communication modes and 
audiences and use available software features to enhance the communicative 
effect of their work. 


1 0.4% 


0.1% 


Level 5 

Students working at level 5 evaluate the credibility of information from 
electronic sources and select the most relevant information to use for a 
specific communicative purpose. They create information products that show 
evidence of planning and technical competence. They use software features 
to reshape and present information graphically consistent with presentation 
conventions. They design information products that combine different 
elements and accurately represent their source data. They use available 
software features to enhance the appearance of their information products. 


\ 11.9% 


7.7% 


Level 4 

Students working at level 4 generate well targeted searches for electronic 
information sources and select relevant information from within sources to 
meet a specific purpose. They create information products with simple linear 
structures and use software commands to edit and reformat information 
products in ways that demonstrate some consideration of audience and 
communicative purpose. They recognise situations in which ICT misuse may 
occur and explain how specific protocols can prevent this. 


48.9% \ 


/40.8% 


Level 3 

Students working at level 3 generate simple general search questions and 
select the best information source to meet a specific purpose. They retrieve 
information from given electronic sources to answer specific, concrete 
questions. They assemble information in a provided simple linear order to 
create information products. They use conventionally recognised software 
commands to edit and reformat information products. They recognise 
common examples in which ICT misuse may occur and suggest ways of 
avoiding them. 


/ 32.0% 

1 d 


38.s\ 


Level 2 

Students working at level 2 locate simple, explicit information from within a 
given electronic source. They add content to and make simple changes to 
existing information products when instructed. They edit information 
products to create products that show limited consistency of design and 
information management. They recognise and identify basic ICT electronic 
security and health and safety usage issues and practices. 


1 6.4% 


12.6% \ 


Level 1 

Students working at level 1 perform basic tasks using computers and 
software. They implement the most commonly used file management and 
software commands when instructed. They recognise the most commonly 
used ICT terminology and functions. 


1 0.4% 



Year 6 Year 10 



6p: Year 6 proficient standard 
lOp: Year 10 proficiient standard 
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The separation of Year 6 and Year 10 students is shown in TableESl. Only eight 
per cent of Year 6 students performed at Level 4 or above compared to 61 per 
cent of Year 10 students. In contrast 51 per cent of Year 6 students performed 
at Level 2 or below compared to seven per cent of Year 10 students. 



Proficient Standards in ICT Literacy 

In addition to deriving the ICT Literacy proficiency scale, proficient standards 
were established for Year 6 and Year 10. The proficient standards represent 
points on the proficiency scale that represent a ‘challenging but reasonable’ 
expectation for typical Year 6 and 1 0 students to have reached by the end of each 
of those years of study. The proficient standards were established as a result 
of two-day consultations with ICT education experts and representatives from 
all states and territories and all school sectors. The groups included currently 
practising teachers with specific ICT expertise, ICT curriculum experts and 
educational assessment experts. 

The proficient standard for Year 6 was defined as the boundary between levels 

2 and 3 or a score of 375 on the ICT literacy scale. Lorty-nine per cent of Year 6 
students reached or exceeded the Year 6 proficient standard by demonstrating 
the ability to “generate simple general search questions and select the best 
information source to meet a specific purpose, retrieve information from 
given electronic sources to answer specific, concrete questions, assemble 
information in a provided simple linear order to create information products, 
use conventionally recognised software commands to edit and reformat 
information products”. 

The proficient standard for Year 10 was defined as the boundary between levels 

3 and 4 or a score of 500 on the ICT literacy scale and 61 per cent of Year 10 
students reached or exceeded the Year 1 0 proficient standard by demonstrating 
the ability to “generate well targeted searches for electronic information 
sources and select relevant information from within sources to meet a specific 
purpose, create information products with simple linear structures and use 
software commands to edit and reformat information products in ways that 
demonstrate some consideration of audience and communicative purpose”. 



Patterns of ICT Literacy 

Australia’s national goals for schooling assert that: students’ outcomes from 
schooling should be free from the effects of negative forms of discrimination 
based on sex, language, culture and ethnicity, religion or disability; and of 
differences arising from students’ socio-economic background or geographic 
location (MCLLTYA, 1999: Goal 3.1). 
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ICT literacy was quite strongly associated with socioeconomic background. 
Approximately two-thirds (68%) of Year 6 students whose parents were “senior 
managers and professionals” attained the proficient standard compared to 
approximately one-third (32%) of students whose parents were in “unskilled 
manual, office and sales” occupations. There was a gap of 87 scale points 
between the mean for Year 6 students whose parents were classified as “senior 
managers and professionals” and the mean for students whose parents were 
classified as “unskilled manual, office and sales”. For Year 10 students the 
corresponding gap in mean scale scores was 65 points. Three-quarters (75%) 
of Year 10 students whose parents were “senior managers and professionals” 
attained the proficient standard compared to just less than half (49%) of students 
whose parents were in “unskilled manual, office and sales” occupations. 

Indigenous status is also associated with ICT literacy. The percentages of 
non-lndigenous students attaining the proficient standard for each Year were 
somewhat greater than the percentages of Indigenous students, in Year 6 the 
comparison is 50 per cent compared to 30 per cent, in Year 10 the comparison 
is 62 per cent compared to 35 per cent. The net differences in scale scores 
between Indigenous and non-lndigenous students are 36 scale points at Year 
6 and 50 scale points at Year 10, after allowance is made for the associated 
influence of socioeconomic group and geographic location. This is a smaller 
difference than has been observed in the national assessment of Civics and 
Citizenship and in PISA and TIMSS where the scales can be compared. 

iCT literacy was lower for students from remote locations than for their peers 
from metropolitan locations, in Year 6 52 per cent of students from metropolitan 
locations attained the proficient standard compared to 33 per cent of students 
from remote locations. The corresponding figures for Year 10 were 63 per cent 
and 46 per cent. Those differences remained after allowing for the influence of 
other associated factors. 

There was no statistically significant difference between the sexes in the 
percentage attaining the proficient standard at either Year 6 (the magnitude 
of the difference was seven percentage points in favour of females) or Year 10. 
There was no difference in iCT literacy associated with language background. 

At Year 6, when the comparisons among jurisdictions including confidence 
intervals are considered, there appear to be three groups of jurisdictions in 
terms of ICT literacy. The percentages attaining the proficient standard are 
shown in Table ES2. in Year 6, Victorian students, 58 per cent of whom attained 
the proficient standard performed just significantly above the Australian 
average of 49 per cent. There was no significant difference between the national 
performance and that of students in Tasmania, New South Wales and South 
Australia, in Queensland and Western Australia the proportion of students 
attaining the proficiency level was significantly lower (between 38 and 40 per 
cent) than the Australian average. 
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For the Australian Capital Territory and the Northern Territory the sample sizes 
limit our certainty about the differences even though they are relatively large 
in magnitude. The percentage of students attaining the proficient standard in 
the Australian Capital Territory was not significantly different from the national 
average (although the difference was 10 percentage points). For the Northern 
Territory the percentage of students attaining the proficient standard was 
not significantly different from the national average (although the gap was 13 
percentage points). These differences among States and Territories cannot be 
fully accounted for by differences in social and demographic characteristics. 



Table ES2: Percentages of Students from Each State and Territory attaining the Proficient 
Standard 





Year 6 


Year 10 


0 / 

7o 


Confidence 

Interval 


7o 


Confidence 

Interval 


New South Wales 


50.5 


±6.6 


61.1 


±7.6 


Victoria 


57.9 


±6.3 


66.5 


±4.8 


Queensland 


37.7 


±5.3 


59.5 


±7.4 


South Australia 


51.7 


±5.0 


61.4 


±5.4 


Western Australia 


39.6 


±5.4 


55.8 


±6.1 


Tasmania 


48.9 


±9.0 


56.4 


±6.4 


Northern Territory 


36.0 


±10.0 


48.6 


±13.2 


Australian Capital Territory 


58.4 


±12.5 


65.5 


±11.4 


Australia 


48.6 


±3.0 


61.2 


±3.1 



For Year 10 there was no significant difference between the percentage of 
students attaining the proficient standard in any jurisdiction and the Australian 
average. The range was from 67 per cent of students in Victoria attaining the Year 
10 proficient standard to 49 per cent in the Northern Territory and 56 per cent 
in Western Australia but these differences were not statistically significant. 



Familiarity with ICT Literacy 

There were differences evident in the extent to which students in Years 6 and 
10 had the opportunity to become familiar with computers. Fifty-one per cent 
of Year 6 students whose parents had either “skilled or unskilled trades and 
office” occupations had more than five years experience of using computers 
compared to 59 per cent of those students whose parental occupations were 
“professional or managerial”. Among Year 10 students the corresponding 
percentages were 61 per cent and 68 per cent. There were similar differences 
in home computer usage between socioeconomic groups. 

There were quite large differences in the reported frequency of school computer 
use among Year 10 students. The results in Table ESS indicate a substantial 
variation in usage of computers at school by Year 10 students from the highest 
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using jurisdictions (Tasmania, South Australia and Victoria) to the lowest 
(New South Wales). There was much less variation among jurisdictions at Year 
6 with the only statistically significant difference being between the highest 
using State (South Australia) and the lowest using State (New South Wales). 



Table ESS: Estimated Frequency of Use of Computers at School by Year 10 Students 





Mean days per 
month 


Confidence 

Interval 


New South Wales 


7.0 


±0.6 


Victoria 


11.6 


±1.1 


Queensland 


9.6 


±0.8 


South Australia 


12.0 


±0.8 


Western Australia 


9.6 


±1.0 


Tasmania 


12.7 


±0.6 


Northern Territory 


11.0 


±2.4 


Australian Capital Territory 


9.2 


±0.8 



Familiarity with ICT and student attitudes to using computers is associated 
with ICT literacy scores and contributes to part, but not all, of the variations in 
ICT literacy among students. 

Students vary considerably in the computer applications that they use. Those 
patterns of use differ between Year 6 and Year 10, and between males and 
females. Communication is a frequent use at both Year 6 and Year 10 and using 
the internet to look up information is also a frequent application at both Year 
levels. However, there was much less frequent use of applications that involved 
creating, analysing or transforming information. The lack of use of these types 
of application appears to be reflected in the aspects of ICT literacy that were 
less successfully completed by students. 



Conclusion 

One should not assume that students are uniformly becoming adept because 
they use ICT so widely in their daily lives. The results of the assessment 
survey suggest that students use ICT in a relatively limited way and this is 
reflected in the overall level of ICT literacy. Communication with peers and 
using the internet to look up information are frequent applications but there 
is much less frequent use of applications that involve creating, analysing or 
transforming information. There are substantial differences between Year 6 
and Year 10 suggesting that considerable growth in ICT proficiency takes place 
over these four years. Within each Year level there are differences associated 
with socioeconomic background. Indigenous status and remote geographic 
locations (compared to metropolitan locations). 
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Overall, 49 per cent of Year 6 students attained the proficient standard for 
that Year level by being able to: “generate simple general search questions 
and select the best information source to meet a specific purpose, retrieve 
information from given electronic sources to answer specific, concrete 
questions, assemble information in a provided simple linear order to create 
information products, use conventionally recognised software commands to 
edit and reformat information products”. Sixty-one per cent of Year 10 students 
reached or exceeded the proficient standard for Year 10 by indicating that 
they were able to: “generate well targeted searches for electronic information 
sources and select relevant information from within sources to meet a specific 
purpose, create information products with simple linear structures and use 
software commands to edit and reformat information products in ways that 
demonstrate some consideration of audience and communicative purpose”. 
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Chapter 1 

Introduction to the National ICT 
Literacy Sample Assessment, 
2005 



Recent decades have witnessed the development and pervasive implementation 
of information and communication technologies (ICT) throughout society. This 
development has found expression in the term “the information society”. There 
is consensus that the exchange of information and knowledge through ICT is a 
feature of modern societies. ICT provides the tools for the creation, collection, 
storage and use of knowledge as well for communication and collaboration 
(Kozma, 2003). The development of ICT has changed the environment in which 
students develop skills for life, the basis of many occupations and the way a 
number of social transactions take place. ICT literacy has become important for 
life in modern society and its assessment has become a component monitoring 
student achievement in many educational systems. 



ICT Literacy and School Education 

Australia’s national goals for schooling assert that when students leave school 
they should be: 

confident, creative and productive users of new technologies, particularly 
information and communication technologies, and understand the impact 
of those technologies on society. (MCEETYA, 1999: Goal 1.6). 
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There is similar recognition of the importance of ICT literacy in other countries 
that have recognised the importance of education and training in ICT so 
that citizens can access information and participate in transactions through 
these technologies. In the United Kingdom, the Qualifications and Curriculum 
Authority asserts that ICT “is an essential skill for life and enables learners to 
participate in a rapidly changing world” (QCA, 2007) and ICT is compulsory 
within the National Curriculum (http://www.nc.uk.net). In the United States, 
the National Literacy Act includes the ability to use computers in its definition 
of literacy and many states have programs to monitor student ICT literacy 
(Crawford & Toyama, 2002). 

In some senses ICT literacy in school education is analogous to reading 
literacy in that it is both an end and a means. At school young people learn 
to use ICT and they use ICT to learn. In schools ICT is used as the basis for 
instructional delivery systems to increase skills and knowledge in other 
learning areas; as a tool for accessing resources, communicating, analysing or 
conducting simulations. ICT is sometimes seen as providing foundations for 
changing teaching and learning processes. However, ICT is also used so that 
students can develop ICT skills and knowledge and understand the role of ICT 
in learning, work and society. 

Although this report is about young people learning to use ICT that purpose 
should be seen in a wider context. The Ministerial Council for Education 
Training and Youth Affairs (MCEETYA) through its ICT in Schools Taskforce 
has published an overarching national vision of schools using ICT to improve 
learning, teaching and administration: Contemporary Learning: Learning in an 
Online World (MCEETYA, 2005a). This builds on the changes that have occurred 
in the use of information and communication technologies (ICT) in schools 
in the years since the publication of Learning in an Online World: the School 
Education Action Plan for the Information Economy (MCEETYA: 2000). 



Evaluating ICT literacy 

Australian education authorities, through the Ministerial Council on Education, 
Employment, Training and Youth Affairs (MCEETYA), are committed to assess 
the extent to which the goal of ICT literacy, and other national goals, is being 
achieved. The Performance Measurement and Reporting Taskforce (PMRT) of 
MCEETYA is responsible for the assessment process. Assessments of literacy 
and numeracy are conducted annually using the full population of students at 
Years 3, 5, 7 and (shortly) 9. In science, civics and citizenship and ICT literacy 
assessments are conducted using sample surveys of students in Year 6 and 
Year 10 every three years. This is a report of the first of the sample surveys of 
ICT literacy conducted in October 2005. 

This report outlines the development of a computer-based tool for assessing 
ICT literacy among school students and the application of that tool with a 
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nationally representative sample of 7,400 students from Year 6 and Year 10 
in Australian schools. The report describes the development, validation and 
refinement of a progress map that identifies a progression of ICT literacy. It 
describes the ICT literacy levels of Australian school students overall and for 
particular groups of students. 



What is ICT literacy 

Definition 

For the purpose of this assessment ICT literacy is defined as: 

the ability of individuals to use ICT appropriately to access, manage, 
integrate and evaluate information, develop new understandings, and 
communicate with others in order to participate effectively in society 
(MCEETYA, 2005b). 

The definition draws on the Eramework for ICT Literacy developed by the 
International ICT Literacy Panel in 2003 and the OECD PISA ICT Literacy 
Eeasibility Study (International ICT Literacy Panel, 2002) and is consistent with 
an emerging consensus about ICT literacy (Kelly & Haber, 2006). Definitions 
of ICT literacy differ in the extent to which they emphasise computer skills 
and system knowledge, applications for analysis and information processing 
or communication (e.g. as new literacies or multi-literacies'). They also vary 
in the extent to which they see ICT literacy as a distinct attribute rather than 
embedded in the context of other domains (such as reading or science). The 
MCEETYA definition that is the basis for this assessment sees ICT literacy 
as a distinct attribute but as one that emphasises real-world application to 
relatively high order processes in a range of contexts. 



Scope 

Although ICT can be broadly defined to include a range of tools and systems 
this assessment focuses primarily on the use of computers rather than other 
forms of ICT. There are three reasons for adopting this focus. Eirst, although 
one can envisage computer literacy as a construct, one cannot be sure that the 
same construct would fit a range of different technologies. It is not clear that 
using mobile phones for text messaging is part of the same dimension as using 
a computer program to search for or transform information. Secondly, one 
could reasonably expect all students to have had deliberate and considered 
exposure to computer technology in schools. Other forms of ICT are less 
closely connected to what happens in school and less clearly the business 
of schools. The third is that as a first step in a new field it was important to 
contain the scope so as to make the assessment manageable. Nevertheless, 



1 See, for example, Mioduser, Nachmias and Forkosh-Baruch (2007) 
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the ICT literacy assessment framework acknowledges that the cornerstones of 
ICT literacy are the importance of knowledge and skill regarding contemporary 
technology and the fundamental understandings and aptitudes towards dealing 
with information and learning about new technology. 



Progress in ICT literacy 

Any assessment is underpinned by a concept of progress in the area being 
assessed. This assessment of ICT literacy is based on a hierarchy of what 
students typically know and can do. Progress in ICT literacy is articulated 
in the assessment framework as a progress map that describes six levels of 
increasing complexity and sophistication in using ICT. A progress map is always 
a draft to be developed and refined as a result of the empirical evidence. For 
convenience, students’ skills and understandings are grouped and described 
in levels of proficiency. Each level describes skills and understandings 
that are progressively more demanding. The progress map is a generalised 
developmental sequence that enables information on the full range of student 
performance to be collected and reported. The draft iCT Literacy progress was 
based on three “strands”: 

• working with information; 

• creating and sharing information; and 

• using ICT responsibly. 

in Working with Information, students progress from using keywords to retrieve 
information from a specified source, through identifying search question terms 
and suitable sources, to using a range of specialised sourcing tools and seeking 
confirmation of the credibility of information from external sources. 

in Creating and Sharing Information, students progress from using functions 
within software to edit, format, adapt and generate work for a specific purpose, 
through integrating and interpreting information from multiple sources with 
the selection and combination of software and tools, to using specialised 
tools to control, expand and author information, producing representations of 
complex phenomena. 

in Using ICT Responsibly, students progress from understanding and using basic 
terminology and uses of ICT in everyday life, through recognising responsible 
use of ICT in particular contexts, to understanding the impact and influence of 
ICT over time and the social, economic and ethical issues associated with its 
use. 
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Approach to the Assessment of ICT 
literacy 



ICT assessment 

Internationally there have been four main approaches to the assessment of ICT 
literacy. 

• Pen and paper methods are used and students are asked about ICT without 
any use of the tools themselves. 

• Computer technology is used to deliver questions in traditional forms or 
where computer technology is used only to present stimulus material but 
the questions are multiple choice or constructed response questions that 
do not require any use of the technology other than to record an answer. 

• Simulated ICT screens displayed on a computer so that students are 
required to take an action in response to a question and their response is 
recorded. These are typically single tasks (such as copying, pasting, using 
a web browser) but the screens need to allow for all possible “correct” 
responses to be recognised. The responses can be automatically scored. 

• Students are required to use authentic applications and save the products 
of their work for subsequent assessment by assessors. This approach 
typically involves using multiple applications concurrently (which is 
what one typically does with computer software) to perform larger (but 
specified) tasks and provides the best method of assessing the higher 
levels of ICT literacy. 

• Requiring students to complete authentic tasks in authentic contexts was 
seen as fundamental to the design of the Australian National ICT Literacy 
assessment so the fourth approach was incorporated along with the 
third approach. The inclusion of communication in the ICT construct is 
recognition of the overwhelmingly prevalent context in which students 
develop and demonstrate computer knowledge and skills. More generally, 
it can be noted that ICT holds considerable promise for expanding and 
enriching assessment tools so that they can be based on authentic tasks 
(Pellegrino, Chudowosky & Glaser, 2001). 

The assessment instrument 

One of the challenges for authentic assessments is that of delivery on a large 
scale. Traditional assessments that can be given consistently to large numbers 
of test takers over a large geographic area such as paper or online surveys 
and - in some cases - automated skills-based assessments can provide useful 
information. However, they are limited in the extent to which they can be used 
to analyse complex work products or behaviours such as a student’s ability to 
evaluate and integrate information. Assessment techniques that provide for 
analysing higher-level abilities (involving rubric-scored portfolios or classroom 
observations) have proven to be difficult to administer beyond classroom level 
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because of the resources and the problems of ensuring consistent assessment 
work products and observations. 

This assessment instrument combined multiple item types within a single, 
consistently administered assessment. Within each ICT assessment module 
students were asked multiple-choice questions to assess knowledge, to perform 
specific functions within simulations of software products to assess skill with 
applications such as Microsoft Windows, Word and Internet Explorer, to provide 
constructed responses to specific questions and create work products using 
live applications. The work products, created under controlled, consistent 
conditions for every student, were evaluated using standardised rubrics by 
trained assessors. 



Assessment modules and items 

The assessment instrument consists of seven discrete thematic modules. One 
module, the General Skills Test, includes only simulation and multiple-choice 
assessment items. Six of the modules, the Hybrid Assessment Modules (HAMs), 
integrate conventional simulation, multiple-choice and constructed response 
items with live application software. All students first completed the General 
Skills Test and then two HAMs. One reason for conducting the assessment 
with a number of HAMs is to ensure that the assessment instrument accesses 
what is common to the ICT construct across a sufficient breadth of authentic 
contexts. 

The General Skills Test was created to assess students’ fundamental computer 
skills and knowledge and the item formats used enabled all items to be 
automatically scored by the system. Since all students completed the General 
Skills Test, data from these items could be used as universal links in estimating 
student achievement and test item difficulty on the same scales. The General 
Skills Test also served as a gatepost test, by enabling students with the least 
proficiency to be automatically allocated the two easiest HAMs. 

The HAMs followed a basic structure in which the simulation, multiple- 
choice and short-constructed response items were followed by a single large 
task using at least one live software application. The audience and software 
related communicative context were specified to the students as part of the 
communicative purpose of the large task. 



Administration 

Computer-based assessment 

The assessment was required to be computer based and it was important that 
the computer-based assessment was administered on an environment that was 
uniform for all students on computers that functioned reliably. For both the 
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field trial and the main survey the ICT literacy assessment was administered 
using sets ol six networked laptop computers (five were lor students and one 
was lor the test administrator) with all necessary software installed. Test 
administrators travelled to each school with the networked computers to 
manage the process. 

Field trial 

A field trial was conducted in April 2005. Assessments were obtained Irom 
617 students in 66 schools (the intention was to sample ten students in each 
school) Irom lour States. Students completed three modules each. There were 
275 respondents to the General Skills Test and an average ol 160 respondents to 
each HAM. In the field trial there were 332 Year 6 students (35 schools) and 285 
Year 10 students (31 schools). Overall, 53 per cent were lemale, 3 per cent were 
Indigenous, and 27% had a parent who spoke a language other than English at 
home. Some 84% had used a computer lor three years or more, 54% said they 
used a computer every day at home (and 84% used a computer every week), 
13% said they used a computer every day at school (and 67% every week), and 
more than 90% used a windows-based computer either solely or as well as a 
Macintosh computer at home. The most Irequent uses ol computers were using 
the Internet to look up inlormation, playing games and doing word processing. 
The least Irequent uses ol computers were lor spreadsheets, mathematics 
applications, language or other learning programs, and programming. 



Main survey 

For the main survey, in each school the assessment process involved five 
students in each ol three sessions. In total there were 21 networks (or mini- 
labs) taken into schools by trained administrators. Although the logistics 
ol this operation were challenging it was successlully implemented. At the 
end ol each day the files ol student responses were burned to CD-ROMs and 
despatched to ACER where they were compiled in the data file lor assessment 
and analysis. Greater detail is provided in Appendix 1. 

The sample was a two-stage (probability proportional to size) cluster design 
to ensure that each eligible student had an equal chance ol being selected in 
the sample. Identical procedures were followed for the Year 6 and the Year 10 
samples. In the first stage schools in each stratum were selected, Irom within 
the strata ol State or Territory and sector, with a probability proportional to 
the number ol students in the relevant Year level enrolled at that school. In the 
second stage students (other than those students defined as excluded under 
PMRT protocols) were selected at random. This involved obtaining Irom the 
school a list ol all eligible students in the Year level and selecting a random 
sample Irom the list. Replacement students were selected in case one or more 
ol the students declined to participate or is absent on the day ol testing. 
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The sample design was for a sample of 7,800 students (3,900 at each of Year 6 
and Year 10) from 520 schools (260 at each of Year 6 and Year 10). The achieved 
sample totalled 3,746 Year 6 and 3,647 Year 10 students from 264 primary and 
253 secondary schools across Australia. The participation rates were 96 per 
cent at Year 6 and 93 per cent at Year 10. Greater detail about the sample is 
provided in Appendix 1. The survey took place over a two-month period from 
12 September to 14 November 2005. 



Data management 

Two main forms of assessment data were generated by students using the 
assessment tool. The first were those based on student responses to tasks that 
are either correct or not correct (including the possibility that there could be 
several correct ways of responding to a task) or responses to multiple choice 
items. These were scored automatically by the system and stored directly 
in a student-scores database. The second were those where a student wrote 
a short constructed response or produced an artefact that is compiled for 
scoring by trained assessors. The short constructed responses and artefacts 
were scored by the assessors using detailed rubrics and an on-line marking 
system. 

Each automatically scored and short constructed response item typically 
addressed one specific aspect of the ICT literacy framework. Responses to items 
were therefore scored according to the degree to which they demonstrated 
achievement of the relevant aspect, in some cases, a partial credit scoring 
model was used to deal with responses that were indicative of qualitatively 
different degrees of achievement of an aspect. 

The student artefacts typically provided evidence of achievement across a 
range of aspects of the ICT literacy framework. The artefacts were therefore 
scored using a rubric of criteria in which each criterion related to discrete 
aspects of the framework. Broadly the assessment criteria for the artefacts 
could be classified as relating to either the substantive properties of the 
student work, or the students’ use of the available software features, in each 
case the assessment criteria were couched and considered in terms of the 
overall communicative purpose of the artefact. 

The on-line marking system provided for student artefacts to be displayed on a 
screen and scores to be recorded and included in the student scores database 
with the automatically generated scores. Raters worked as a team with four 
supervisors second-marking a random 10% of all scored student work by all 
raters. Overall the correlation between rater and supervisor scores was 0.86. 

Once the student scores database was assembled, analysis of responses was 
undertaken to establish the psychometric properties of the scales, individual 
analyses were conducted of the full data set, each strand, the General Skills 
Test (GST) module and each module (because the modules are not intended 
to be of equal difficulty). The GST was given special attention because this 



module was common to all students and was used as a basis for assigning 
students to tasks. Analyses were also conducted to test for differential item 
functioning by sex and Year level. The analysis also tested the links (common 
items) between Year 6 and Year 10. Subsequent analysis involved a standards 
setting consultation to establish the level that was deemed a proficient 
standard. Student scores on the ICT Literacy scale were analysed in relation to 
student characteristics and the State or Territory from which they came. 



Structure of This Report 

This chapter of the report provides an introduction to the national sample 
assessment in ICT literacy, it outlines some general issues associated with ICT 
in schools, the assessment of ICT literacy and the way in which the assessment 
was conducted. 

Chapter 2 is concerned with the assessment framework that guided the project 
and the assessment instrument that was used, it describes the elements of iCT 
literacy that constitute the framework and the initial or theoretical progress 
map that guided the development of the assessment instrument, it also 
provides some detail about the modular structure (the general skills test and 
the hybrid modules) of the assessment instrument and the types of items that 
provided the content of the modules. The chapter also outlines the content of 
the student questionnaire that was administered (on computer) at the same 
time as the assessment instrument. 

Chapter 3 describes the ICT literacy of Australian school students, it is built 
around an analysis of item and scale statistics and provides a description of 
the ICT literacy scale. The ICT literacy scale is described in terms of scale 
statistics and a set of proficiency bands with descriptors of each band derived 
from an interpretation of the nature of the items that fall within each band, 
items are mapped to levels on the scale and the described proficiency bands 
characterise the empirically validated progress map. Chapter 3 compares 
the proficiency of Year 6 and Year 10 students on the ICT literacy scale and 
also compares the performance of students at each of those Year levels with 
the proficient standard that was established through a process of structured 
consultation with experts in the field. 

Chapter 4 is concerned with differences in ICT literacy among groups of 
students, it compares the performance of males and females, students from 
the different Sates and Territories and students from specified social groups. 
These social groups are based on socio-economic status, language background. 
Indigenous status, and geographic location. 

Chapter 5 uses data from the student survey to examine students’ experience 
of using computers and the relationship between familiarity with computing 
and performance on the ICT literacy scale, it concludes with a multivariate 
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analysis of the survey data to determine the factors that are most strongly 
related to ICT literacy. 

Chapter 6 provides some inferences from these data on ICT literacy, it provides 
an overview of student performance at Year 6 and Year 10 and discusses some 
implications of the differences between Year 6 and Year 10 and the differences 
in ICT literacy among groups of students. 
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Chapter 2 

Assessing ICT Literacy 



The results of any assessment of ICT literacy depend upon the ways in which the 
concept is defined, the assessment tools that are derived from that conception 
and the way in which the assessment is administered. In this chapter the 
assessment domain that framed the ICT literacy assessment is described 
including the definition adopted, the elements envisaged as comprising ICT 
literacy (sketching the field) and the theory of progress in ICT literacy (that 
outlines a vision of what it means to become more ICT literate). The chapter 
also describes the assessment tool that was developed to be administered 
on computers using tasks that embodied as much authenticity as possible. In 
other words the assessment tool was developed so as to replicate how people 
used computers when approaching real tasks. Results from any assessment 
need to be interpreted in terms of how the assessment was delivered. The ICT 
literacy framework and its subsequent operationalisation as the assessment 
instruments are described in detail in this chapter. In addition the chapter 
describes the sample of nearly 7,400 students, from 517 schools, that completed 
the ICT literacy assessment. The sample was selected to be representative of 
the Australian population of Year 6 and Year 10 students so that valid estimates 
of ICT literacy could be derived for the nation, for States and Territories and 
for designated groups of students. 
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Assessment Domain for ICT Literacy 



Background 

An expert committee was asked to create an assessment domain for ICT 
literacy that could be used to frame this inaugural and the ongoing three-yearly 
national sample assessments of ICT literacy. The assessment domain is the 
result of an ‘extensive literature review and examination of international and 
national surveys, panels and frameworks’ (MCEETYA, 2005b). The assessment 
domain includes: 

• the definition of ICT literacy; 

• a description of the ICT literacy domain, strands and the progress map; 

• the types of items that will be used in ICT literacy assessment; and 

• how the results from the assessments will be reported (MCEETYA, 2005b). 



Defining ICT literacy 

In July 2001, MCEETYA agreed to define ICT as ‘technologies used for 
accessing, gathering, manipulation and presentation or communication of 
information’ (MCEETYA, 2005b). For the purpose of this, the first national 
sample assessment of ICT literacy, it was decided that the assessment of ICT 
literacy would focus on students’ use of computer tools. For the purpose of the 
National Assessment Program, ICT literacy is defined as: 

the ability of individuals to use ICT appropriately to access, manage, 
integrate and evaluate information, develop new understandings, and 
communicate with others in order to participate effectively in society 
(MCEETYA, 2005b). 

Key processes of ICT literacy 

The assessment domain describes ICT literacy as comprising the following six 
processes: 

1. Accessing information - identifying the information needed and knowing 
how to find and retrieve information; 

2. Managing information - organising and storing information for retrieval and 
reuse; 

3. Evaluating - reflecting on the processes used to design and construct ICT 
solutions and about making judgements regarding the integrity, relevance 
and usefulness of information; 

4. Developing new understandings - creating information and knowledge by 
synthesising, adapting, applying, designing, inventing or authoring; 

5. Communicating with others - exchanging information by sharing knowledge 
and creating information products to suit the audience, the context and 
the medium; and 
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6. Using ICT appropriately - making critical, reflective and strategic ICT 
decisions and about using ICT responsibly by considering social, legal and 
ethical issues. 

Draft ICT literacy progress map 

The assessment domain included a draft progress map of student achievement 
in ICT literacy. The draft progress map, shown in Figure 2.1, describes the 
assumed ‘typical’ growth of students’ ICT knowledge, understandings and skills. 
The draft progress map has been a key reference for both the development of 
the assessment items and the consequent construction of the ICTL scale. The 
progress of student achievement in the ICT Literacy processes can only be 
demonstrated with consideration of the communicative context, purpose and 
consequences of the medium. As such, the ICT Literacy progress map is based 
on three strands: 

• Strand A - Working with information 

• Strand B - Creating and sharing information 

• Strand C - Using ICT responsibly 

In each of the strands there are six proficiency levels hypothesised. These are 
not proposed as discrete steps that are discontinuous but are proposed as a 
means of representing progress within each strand. Table 2.1 includes the three 
strands and the six levels within each strand. It is proposed as a representation 
of the field and does not assume that the strands are empirically distinct. The 
assessment instrument is designed so that approximately 80 per cent of the 
total assessment content is distributed evenly between Strands A and B and 
the remaining 20 per cent to Strand C. 

In Working with Information, students progress from using keywords to retrieve 
information from a specified source, through identifying search questions, 
terms and suitable sources, to using a range of specialized sourcing tools and 
seeking confirmation of the credibility of information from external sources. 

In Creating and Sharing Information, students progress from using functions 
within software to edit, format, adapt and generate work for a specific purpose, 
through integrating and interpreting information from multiple sources with 
the selection and combination of software and tools, to using specialized 
tools to control, expand and author information, producing representations of 
complex phenomena. 

In Using ICT Responsibly, students progress from understanding and using basic 
terminology and uses of ICT in everyday life, through recognizing responsible 
use of ICT in particular contexts, to understanding the impact and influence of 
ICT over time and the social, economic and ethical issues associated with its 
use. 
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Table 2.1: Information and Communication Technology Literacy Draft Progress Map 

ICT literacy is the ability of individuals to use ICT appropriately to access, manage and evaluate 
information, develop new understandings, and communicate with others in order to participate 
effectively in society. 





Strand A: Working with 
Information 


Strand B: Creating and 
Sharing information 


Strand C: Using ICT 
responsibly 




This strand includes 
identifying the information 
needed; formulating and 
executing a strategy to 
find information; making 
judgements about the 
integrity of the source and 
content of the information; 
and organising and storing 
information for retrieval and 
reuse. 


This strand includes: 
adapting and authoring 
information; making 
choices about the nature 
of the information product; 
reframing and expanding 
existing information to 
develop new understandings; 
and collaborating and 
communicating with others. 


This strand includes: 
understanding the capacity 
of ICT to impact on 
individuals and society, 
and the consequent 
responsibility to use and 
communicate information 
legally and ethically. 


6 


Uses a range of specialised 
sourcing tools. Seeks 
confirmation of the integrity 
of information from credible, 
external sources. Uses tools, 
procedures and protocols 
to secure and retrieve 
information. 


Uses specialised tools to 
control, expand and author 
information. Produces 
complex products. Critiques 
work and applies knowledge 
of conventions that shape 
interpretations when 
communicating across a 
range of environments and 
contexts. 


Understands the impact and 
influence of ICT over time, 
recognising the benefits, 
constraints and influence 
of social, legal, economic 
and ethical issues on 
participation in society. 


5 


Searches for and reviews 
the information needed, 
redefining the search to limit 
or expand. Judges the quality 
of information for credibility, 
accuracy, reliability and 
comprehensiveness. Uses 
appropriate file formats and 
procedures to store, protect, 
retrieve and exchange 
information. 


Uses tools to interrogate, 
reframe and adapt 
information. Uses a range 
of tools to create and 
enhance the design, style 
and meaning of information 
products to suit the purpose 
and audience. 


Understands the social, 
legal, economic and ethical 
consequences associated 
with using ICT across a 
range of environments and 
contexts. 


4 


Develops questions or 
keyword combinations and 
selects appropriate tools to 
locate information. Appraises 
located information for 
relevance, currency and 
usefulness. Uses tools 
to structure, group and 
reorganise information for 
retrieval. 


Integrates and interprets 
information from multiple 
sources. Selects and 
combines software and 
tools to structure, link and 
present work. Communicates 
work for different purposes, 
environments and contexts. 


Understands the need for 
laws, codes of conduct 
and procedures for ICT 
use in different contexts. 
Recognises the potential for 
misuse of ICT and that there 
are procedures to address 
this. 


3 


Identifies a search question, 
terms and suitable sources. 
Browses and retrieves 
information. Compares and 
contrasts information from 
similar sources. Organises 
and arranges relevant 
information and files. 


Reorganises information 
from similar sources, using 
the main ideas. Selects 
software and tools to 
combine and transform text, 
images and other elements. 
Communicates work using 
different representations for 
particular contexts. 


Recognises fair use, software 
restrictions and legal 
requirements. Identifies 
responsible use of ICT in 
particular contexts. 


2 


Identifies and uses keywords 
in a search to locate and 
retrieve information from 
various sources. Identifies 
and records relevant content. 


Uses the functions within 
software to edit, format, 
adapt and generate work to 
achieve a specific purpose 
and when communicating 
with others. 


Identifies codes of conduct 
and ergonomic practices 
for ICT. Understands ICT 
terminology and use of 
computers in society. 


1 


Uses keywords provided to 
retrieve information from 
a single, specified source. 
Recognises information 
required. Opens software and 
saves files. 


Identifies and uses some 
of the basic symbols and 
functions of software to 
record ideas. 


Understands and uses basic 
terminology and general 
procedures for ICT. Describes 
uses of ICT in everyday life. 
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The ICT Literacy Assessment Instrument 



Platform 

The assessment was required to be computer based and it was important that 
the computer-based assessment was administered on an environment that was 
uniform for all students on computers that functioned reliably. For both the 
field trial and the main survey the ICT literacy assessment was administered 
using sets of six networked laptop computers (five were for students and one 
was for the test administrator) using MS Windows operating systems and with 
all necessary software installed. 

The software installed on each computer contained all the assessment modules 
and a management system that confirmed the identity of the selected student, 
asked basic registration information, assigned each student to the modules 
appropriate to their Year level (this was random within each Year level for 
students who demonstrated minimum competence on the initial module) and 
collected student responses to the survey questions^. 

The on-screen environment of the assessment instrument had three main 
sections: a surrounding border of test-taking information and navigation 
facilities; a central information section that can house stimulus materials for 
students to read or (simulated or live) software applications; and a lower 
section containing the instructional and interrogative text of the assessment 
items and the response areas for multiple-choice and constructed response 
items. The environment as seen by students is represented in Figure 2.1. 





Figure 2.1: On-Screen Environment for ICT Literacy Assessment 2005 



2 The assessment instrument package integrated software from four different providers on 
a Microsoft Windows XP platform. The two key components of the software package were 
developed by First Advantage Assessment Solutions (formerly SkillCheck) (Boston, MA) 
and SoNet Software (Melbourne, Australia). The First Advantage system provided the test 
management software responsible for delivering the assessment items and capturing student 
data. It also provided the simulation, short constructed response and multiple choice item 
platforms. The SoNet software enabled live software applications (such as Microsoft Word) to 
be run within the global assessment environment and for the resultant student products to be 
saved for later assessment. 
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So as to ensure the smooth operation of the system and to assure data quality, 
test administrators travelled to each school with the networked computers 
to manage the process. The assessment was administered to groups of five 
students in each of three testing sessions during the school day. 

Structure of the instrument 

The ICT assessment instrument was designed to model students’ typical 
‘real world’ use of ICT. Task authenticity was included in the ICT assessment 
instrument in two main ways. Firstly, students completed all tasks on computer 
using a seamless combination of simulated and live software applications. 
Secondly, the assessment items were grouped in thematically linked modules 
each of which followed a linear narrative sequence. The narrative sequence in 
each module typically involved students collecting and appraising information 
before synthesising and reframing the information to suit a particular 
communicative purpose and given software genre. The overarching narratives 
across the modules covered a range of school-based and out-of-school based 
themes. The assessment items were presented in a linear sequence to students. 
Students were not permitted to return to previously completed items as, in 
some cases, later items in a sequence provide clues or even answers to earlier 
items. 



Assessment item types 

The elements of the integrated software systems are each suited to accessing 
different aspects of the ICT assessment construct. The conventional simulation, 
short constructed response and multiple choice item platforms were suited to 
assessing ICT knowledge and discrete skills and capturing students’ analytical 
responses to assessment stimulus materials such as information on websites. 
The live software integrated in the assessment package enables students to 
complete a range of authentic ICT products. 

There were five distinct types of assessment items or tasks in the ICT literacy 
assessment instrument. The item type used for each item was determined 
by the substance of the item and the capacity of the available software to 
manage the full functionality of the item. It was neither necessary nor possible 
to predetermine the proportion of item types within each module or across 
the assessment instrument as a whole. The different types of items access 
different types of student achievement information across the three ICT 
literacy strands. The item types, the type of information they access and their 
technical properties are summarised in Table 2.2. 
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Table 2.2: Summary of ICT Literacy Assessment Task Types, Information Accessed and 
Technical Properties 



Item/Task Types 


Information Accessed 


Software Type and 
Response Protocol 


Scoring 


Multiple-choice 
questions (MCQ) 


Knowledge and 
understandings of ICT 
literacy across the three 
strands 


Static information screen 
with MCQ response 
section; student 
responses recorded in 
individual student data- 
files 


Automated 


Simple software 
skills performance 
tasks 


Capacity to complete 
simple (one or two step) 
software and system 
management tasks 
(mainly strands A and B) 


Simulation; student 
responses recorded in 
individual student data- 
files 


Automated 


Short constructed 
responses 


Knowledge and 
understandings of ICT 
literacy across the three 
strands 


Static information 
screen with constructed 
response field; student 
responses saved as 
text fields in individual 
student data files 


Manual - human 
scored 


Complex software 
skills performance 
tasks 


Capacity to complete 
complex (multi-stage) 
software tasks (mainly 
strands A and B) 


Live single application; 
student responses saved 
as uniquely labelled 
software application files 
(e.g. ‘*.doc, *.xls) 


Manual - human 
scored 


Large tasks 


Combined knowledge 
and understandings 
of ICT literacy across 
the three strands with 
the capacity to create 
complex information 
products across a range 
of software types 


Simultaneously available 
live application files; 
student responses saved 
as uniquely labelled 
software application files 
(e.g. ‘*.doc, *.xls) 


Manual - human 
scored against 
multiple 

assessment criteria 



The assessment modules 

The assessment instrument consisted of seven discrete thematic modules. 
One module, the General Skills Test, included only simple software skills 
performance and multiple-choice assessment items. The General Skills Test 
was completed by all students. Six of the modules, the Hybrid Assessment 
Modules (HAMs) integrated all item types. 

All students first completed the General Skills Test and then two HAMs. One 
reason for conducting the assessment with a number of HAMs is to ensure 
that the assessment instrument accesses the content suggested by the draft 
progress map across a sufficient breadth of authentic contexts. Figure 2.2 
shows the workflow from registration through assessment to completion. 
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Figure 2.2: Workflow through the ICT literacy assessment 



The General Skills Test 

The General Skills Test serves two main purposes in the assessment instrument. 
First, as all students completed the General Skills Test, data from these items 
can be used as universal links in estimating student achievement and test item 
difficulty on the same scale. Second, the General Skills Test is designed to 
be a gatepost test of basic computer proficiency. The content of the General 
Skills Test was created to assess students’ fundamental computer skills and 
knowledge and the item formats used enabled all items to be automatically 
scored. A cut-score on the General Skills Test was established using data from 
the field trial. Students achieving less than the cut-score were deemed to have 
insufficient ICT capacity to cope with the demands of the more difficult HAMs. 
These students were automatically allocated the two easiest HAMs. 



The Hybrid Assessment Modules 

Students who demonstrated at least basic proficiency on the General Skills 
Test were randomly allocated any two Grade level appropriate HAMs. in the 
final survey, approximately 90 per cent of Year 6 and 99 per cent of Year 10 
students demonstrated basic proficiency on the General Skills Test. 

Each HAM had a single unifying theme. Five of the six HAMs followed a basic 
structure in which the software skills performance, multiple-choice and short- 
constructed response items form the lead up to a single large task using at least 
one live software application^. Typically the lead-up tasks require students to: 
manage files; perform simple software functions (such as inserting pictures 
into files); search for information; collect and collate information; evaluate and 
analyse information; and perform some simple reshaping of information (such 
as drawing a chart to represent numerical data) . The large tasks that provide the 
global purpose of five of the six HAMs are then completed using live software. 
When completing the large tasks, students typically need to select, assimilate 



3 The module entitled “help desk” involved students moving back and forth between different 
types of task. 
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and synthesise the information they have been working with in the lead-up 
tasks and reframe the information to fulfil a specified communicative purpose. 
The audience and software related communicative context are specified to the 
students as part of the communicative purpose of each large task. Students 
spent between 40 per cent and 50 per cent of the time allocated for the module 
on the large task. The modules with their associated large tasks are shown in 
Table 2.3. 



Table 2.3: Hybrid Assessment Modules and Large Tasks 



Module 


Large task 


Flag Design 
(Year 6) 


Students use purpose-built previously unseen flag design graphics 
software to create a flag. 


Photo Album 
(Year 6 & 10) 


Students use unseen photo album software to create a photo album to 
convince their cousin to come on holiday with them, 


DVD Day 
(Year 6 & 10) 


Students navigate a closed web environment to find information and 
complete a report template. 


Conservation 
Project (Year 6 & 10) 


Students navigate a closed web environment and use information 
provided in a spreadsheet to complete a report to the Principal using 
Word. 


Video Games and 
Violence 
(Year 10) 


Students use information provided as text and empirical data to create 
a PowerPoint presentation for their class. 


Help Desk 
(Year 6 & 10) 


Students play the role of providing general advice on a community Help 
Desk and complete some formatting tasks in Word, PowerPoint and 
Excel. 



Four of the six HAMs were undertaken by both Year 6 and Year 1 0 students of all 
abilities, one was undertaken by Year 10 students only. One HAM, Flag Design, 
was taken by Year 6 students of all abilities and only by Year 10 students who 
demonstrated below basic proficiency on the General Skills Test. 



Student Background Questionnaire 

The student background questionnaire consisted of questions concerned with 
students’ access to, experience and use of computer technology, attitudes 
to computers and background characteristics. Questions about access, 
experience and attitudes were included specifically in this assessment of ICT 
literacy so as to be able to interpret better patterns of performance. Questions 
about background characteristics were included so as to be able to report the 
distribution of ICT literacy across the student population. 

The questions about access to, experience of and use of computer technology 
were based on questions asked in the computer familiarity section of the PISA 
2003 student survey and were concerned with: 

• the length of time for which students had been using computers; 

• the types of computer (windows, apple and other) used at school, home 
and other places; 
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• the frequency with which students used a computer in each of these 
locations; 

• the frequency with which students used a computer for specified education- 
related functions (look up information on the internet, word processing 
spreadsheets, mathematics, language or other learning programs on a 
computer, programming); and 

• the frequency with which students used a computer for specified 
entertainment-related functions (downloading games or music, playing 
games, using drawing, painting or graphics programs, email or “chatting”, 
listening to music or watching DVDs). 

The questions about attitudes asked students how important it was to them 
to work with a computer, whether working with a computer was fun, how 
interested they were in using computers and whether they lost track of time 
when working with a computer. They were also adapted from the PISA survey 
of computer familiarity. 

The questions about background characteristics were intended to identify 
groups of interest and followed the format used in other sample surveys and 
consistent with the PMRT Data Implementation Manual. The questions were 
asked in formats for presentation and response by computer utilising checking 
of answer boxes, drop-down menus and text boxes for written responses to be 
coded later. The background questions asked about the sex of the respondent, 
the postcode of the student’s permanent home address and the name of the 
location (so as to code geographic location), age (in years and months). 
Indigenous status, country of birth, parental occupation, parental education 
and language spoken at home. 



Sample 

Sample design 

The sampling procedure followed the cluster sampling procedures established 
for national sample surveys conducted by the Performance Measurement and 
Reporting Taskforce. Cluster sampling is cost-effective because a larger group 
of students from the same school can be surveyed at the same time, rather 
than possibly just one or two students if a simple random sample of students 
from the population were to be drawn. Sampling involves a two-stage process 
to ensure that each eligible student has an equal chance of being selected in 
the sample. 

In the first stage schools are selected from a list of all schools in each State 
or Territory with a probability proportional to the number of students in the 
relevant Year level enrolled at that school. Within this process the list of schools 
is explicitly stratified by location and sector and implicitly listed in postcode 
order to ensure that the sample was representative. A small number of schools 
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was excluded from the selection process^. The number of schools from each 
of the mainland States and Territories was similar so as to ensure a similar 
level of precision in the estimates derived from those samples. The percentage 
of schools selected from within Tasmania, the Northern Territory and the 
Australian Capital Territory was greater than would have been expected on a 
proportionate basis so as to improve the precision of the estimates for those 
jurisdictions. 

in the second stage, 15 students were selected at random from a school- 
provided list of all eligible students from the Year level®. At the same time a 
list of replacement students was selected in case one or more of the students 
declines to participate or is absent on the day of testing. By selecting students 
at random from the Year level, and by selecting only 15 students per school, 
the sample had enhanced precision over a sample of the same number of 
students based on selecting intact classes because the effects of students 
being in classes similar to each other was reduced®. 

The total achieved sample for the survey consisted of 7,373 students of which 
3,746 were from Year 6 and 3,627 were from Year 10. Table 2.4 records the 
distribution of the sample across the States and Territories for each Year 
level. 



Table 2.4: Numbers of Students and Schools in the Achieved Sample 





Year 6 


Year 10 


Schools 


Students 


Schools 


Students 


New South Wales 


38 


534 


39 


541 


Victoria 


40 


575 


39 


593 


Queensland 


41 


574 


39 


562 


South Australia 


41 


591 


40 


581 


Western Australia 


41 


570 


40 


557 


Tasmania 


31 


447 


30 


428 


Northern Territory 


16 


231 


11 


162 


Australian Capital Territory 


16 


224 


15 


203 


Total Sample 


264 


3746 


253 


3627 



4 School exclusions are categorised as very remote schools; schools with fewer than five students 
at the Year level, schools for students with intellectual disabilities or migrant language centres. 
School exclusions amounted to fewer than two per cent of schools at Year 6 and less than one 
per cent of schools at Year 10. In Year 10 no jurisdiction had more than three per cent of its 
schools in the excluded category. At Year 6, 25 per cent of listed Northern Territory schools 
were excluded on the basis of size and very remote location but this represented a small 
percentage of students. 

5 Certain students are defined as excluded under PMRT protocols (e.g. students with physical 
or intellectual disabilities, or limited language skills such that they are unable to participate in 
the assessment). 

6 Technically this is known as the “design effect”. It arises because students tend to be grouped 
in schools and classes with other students who are similar to themselves and reduces the 
statistical power of the sample. 
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Table 2.5: Distribution of Weighted Sample Characteristics 





Year 6 


Year 10 




o/ 

/O 


Valid % 


o/ 

/o 


Valid % 


Student Sex 


Boy 


50.9 


51.0 


52.1 


52.2 


Girl 


48.9 


49.0 


47.6 


47.8 


Total 


99.9 


100.0 


99.7 


100.0 


Missing 


0.1 




0.3 




Parental occupation 


Senior managers & professionals 


13.5 


14.4 


16.9 


17.9 


Other managers associate 
professionals 


29.3 


31.2 


36.9 


39.1 


Skilled trades, clerical & sales 


27.3 


29.1 


25.5 


27.0 


Unskilled manual, office & sales 


23.7 


25.3 


15.0 


15.9 


Total valid responses 


93.7 


100.0 


94.3 


100.0 


Not in paid work for 12 months 


3.0 




1.9 




Missing 


3.3 




3.9 




Indigenous Status 


Non Aboriginal or Torres Strait 
Islander 


92.1 


93.5 


94.8 


96.9 


Aboriginal or Torres Strait 
Islander 


6.4 


6.5 


3.0 


3.1 


Total 


98.5 


100.0 


97.8 


o 

o 

o 


Missing 


_U5^| 




■r^.2 m 




Language at home 


English 


73.7 


74.3 


72.3 


73.7 


Other than English 


25.4 


25.7 


25.8 


26.3 


Total 


99.1 


100.0 


98.1 


100.0 


Missing 


0.9 




1.9 




Main Language - Country of birth 


English (including Australia) 


93.7 


94.5 


89.5 


91.3 


Other than English 


5.4 


5.5 


8.5 


8.7 


Total 


99.1 


100.0 


98.1 


100.0 


Missing 


0.9 




1.9 




Geographic location 


Metropolitan 


66.9 


68.0 


69.7 


71.6 


Provincial 


30.0 


30.5 


25.7 


26.4 


Remote 


1.4 


1.4 


1.9 


1.9 


Total 


98.4 


100.0 


97.3 


100.0 


Missing 


1.6 




2.7 
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For the analyses that are used to make population inferences a weighting 
procedure was used. Weighting adjusts for intended design differences in 
the sampling ratios^ and for differential participation®, in this report the 
results of weighted results are recorded so that populations are appropriately 
represented. Table 2.5 records the distribution of social and demographic 
characteristics in the weighted sample. 

Table 2.5 also shows that there were few missing data on any of the 
characteristics. There were missing data for parental occupation from four per 
cent of respondents, for Indigenous status of two per cent of respondents, for 
geographic location of two per cent of respondents, for language background of 
one per cent of respondents and very few for sex. Data for parental education 
have not been reported because of the high levels of respondents who indicated 
that they did not know (33% of Year 6 and 13% of Year 10) in addition to those 
who did not answer the question (2% of Year 6 and 3% of Year 10). More detail 
about the sample is provided in Appendix 2. 



Calculating the precision of estimates 

For any survey there is a level of uncertainty regarding the extent to which 
an estimate measured from the sample of students is the same as the true 
value of the parameter for the population. An estimate derived from a sample 
is subject to uncertainty because the sample may not reflect the population 
precisely, if a statistic was estimated from different samples drawn from the 
same population of students the observed values for the statistic would vary 
from sample to sample. The extent to which this variation exists is expressed 
as the confidence interval. The 95 per cent confidence interval is the range 
within which the estimate of the statistic based on repeated sampling would 
be expected to fall for 95 of 100 samples drawn. The difference between two 
estimates is considered statistically significant at the five per cent level if the 
confidence intervals of those estimates do not overlap. 

The magnitude of the confidence interval can be estimated using formulae 
based on assumptions about the distribution of the measure being considered 
(typically assuming a normal distribution), from modelling based on 
assumptions about the distributions of different levels of clustering in the 
sample or from empirical methods that examine the actual variation in the 
sample. 

The survey sample design in this study involves clustering, stratification, 
and disproportionate allocation which means that it is not appropriate to use 
the estimates of confidence intervals through standard software procedures 
because these generally assume a simple random sample and will therefore 
underestimate the real confidence intervals. The estimates of confidence 



7 This is because students from smaller states or territories are sampled at a greater rate than 
students in larger states to ensure similar numbers and similar precision for all states. 

8 Because some groups of students are more prone to not participate this ensures that results 
are not biased by differences in participation among schools. 
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intervals in this report are based on ‘Jacknife’ replication methods. In 
replication methods a series of sub-samples is derived from the full sample, 
and the statistic of interest is generated for each sub-sample (OECD, 2005: 
174 - 184). The variance is then estimated by calculating the variability in the 
estimate between these sub samples. This technique generates an estimate of 
the standard error of the estimate and the confidence interval is 1.96 times the 
standard error. 



Summary 

The national assessment of ICT literacy was based on a clearly articulated 
definition of the concept that was consistent with international practice. This 
definition was elaborated in terms of key elements that made up three strands 
of a draft progress map that postulated the levels through which students 
would be expected to progress in ICT. The progress map formed the basis of 
the assessment that was developed to be administered on identical computers 
in a proctored environment to students in Year 6 and Year 10. The assessment 
instrument included different types of item including some simulated screens 
and some authentic tasks that used “real” software applications. The items 
were organised in thematic modules designed to represent different contexts 
with any individual student completing one common module and two modules 
assigned at random from a set of six. The achieved sample was not biased and 
represented the major categories of student in appropriate numbers. 
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Chapter 3 

A Profile of ICT Literacy 



The responses of 7,373 students to 227 assessment items (or strictly speaking 
227 possible score points associated with a smaller number of tasks) from 
the seven assessment modules (the GST and the HAMs) provide the basis 
for establishing a profile of ICT literacy. This chapter outlines that process, 
establishes a profile based on six levels of proficiency and reports the 
distribution of students from Year 6 and Year 10 over that profile. The key to the 
process is analysis using item response modelling (the Rasch model) through 
which it is possible to analyse the pattern of student responses (which items 
and how many items they successfully completed) to establish students’ ICT 
literacy level in relation to the test and the difficulty of each item (based on the 
proportion of students who successfully complete each item). This process 
is also the key to generating a single achievement scale on which the items 
from each of the different assessment modules can be located. This is feasible 
because a large number of students completed every possible combination of 
modules; each student completed three of the seven HAMs and all students 
completed the GST. On the basis of the scaled map of item difficulties it is 
possible to describe proficiency levels that provide a generalised description of 
the typical ICT achievements that can be expected of students at each level. 
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Establishing an ICT Literacy Scale 

The analysis that was conducted using the Rasch model is based on the 
property that the chance that a student will answer an item correctly depends 
on their ability and the difficulty of the item®. The analysis results in a single 
continuous scale on which it is possible to locate students according to their 
ICT literacy and assessment items according to the degree of ICT literacy 
required to complete the item. A student placed at a certain point on the ICT 
literacy scale would most likely be able to successfully complete tasks at or 
below that location, and increasingly be more likely to complete tasks located 
at progressively lower points on the scale, but would be less likely to be able to 
complete tasks above that point, and increasingly less likely to complete tasks 
located at progressively higher points on the scale. 

it is possible to illustrate the relationship between the difficulty of tasks and the 
ICT literacy of students using examples shown in Figure 3.1. The first example 
task required students to “paste text from the clipboard into a document”. This 
was completed successfully by 90 per cent of all students (Year 6 and Year 10 
combined). The second example task is one in which students were required 
to “apply a style heading to a paragraph in a document”. This was successfully 
completed by 60 per cent of all students. The third example task is one in which 
students were required to “sort data in a spreadsheet according to specific 
criteria”. This was a relatively difficult item and was completed successfully 
by 18 per cent of all students. 

in relation to these tasks we can consider the performance of three hypothetical 
students. Student A has a high level of ICT literacy, student B has a moderate 
level of ICT literacy and student C has a low level of ICT literacy. 

Using these data it could be concluded that: 

• Student A would typically be able to complete tasks 1 and 2 successfully 
and probably task 3 as well. 

• Student B would typically be able to complete task 1 successfully and 
probably task 2 as well but would be unlikely to be able to complete 
task 3. 

• Student C would be unlikely to be able to complete task 3 or task 2 but 
could probably complete task 1. 



9 In this case, ability and difficulty refer to students’ ICT ability according to the assessment items 
developed to represent the substance of the ICT assessment framework and the amount of ICT 
difficulty required for students to satisfactorily complete each assessment item respectively. 
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ICT Literacy Scale 



Task 3: sort data 
in a spreadsheet 



Student A Student A would be able to 
complete tasks 1 and 2 
successfully and probably 
task 3 as well. 



Task 2: apply style 
heading to paragraph 



Student B Student B would be able to 
complete task 1 
successfully and probably 
task 2 as well but would be 
unlikely to be able to 
complete task 3 



Task 1: paste text 
into document 



Student C Student C would be 
unlikely to be able to 
complete task 3 or task 2 
but could probably 
complete task 1 



Figure 3.1: Relationship between tasks and student performance on the ICT literacy scale 

Figure 3.2 is a result of an analysis of which items were successfully completed 
by each student. It depicts the relationship between student ICT literacy and 
the assessment tasks. The relationship between the student and the tasks is 
based on the probability that a student will complete a task correctly. The 
scale is expressed in “logits” (the logarithm of the ratio of the odds that the 
task would be performed successfully compared to not completing the task 
successfully). This scale empirically operationalises the fundamental concept 
illustrated in Figure 3.1 that if a student is located at a point above a task, the 
likelihood that the student can successfully complete that task is relatively 
high, and if the student is located below the task, the likelihood of success for 
that student on that task is relatively low. 
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Person-Item Location Distribution 

(Grouping set to interval length of 0.20 making 55 Groups) 
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Figure 3.2: Student ICT Literacy Scores and ICT Literacy Item Difficulties 
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Person-Item Location Distribution 

(Grouping set to interval length of 0.20 making 55 Groups) 



0 . 0 % 4 . 0 % 7 . 0 % 




PERSONS ITEMS 

Frequency Frequency 



Level 


No. 


Mean 


SD 


Year 6 


(3746 


-0.295 


1.05 


Year 10 


(3647) 


1.294 


0.91 



Figure 3.3: Student ICT Literacy Scores for Year 6 and Year 10 and Task Difficulties 



Figure 3.2 indicates that the tasks in the ICT literacy assessment spanned a 
wide range of difficulties and that the range of task difficulties was appropriate 
for the spread of ICT literacy among students in Year 6 and Year 10. 
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Differences between Year 6 and Year 10 
Students 



Figure 3.2 records data on the ICT literacy of all students in the survey 
combining Year 6 and Year 10. It is of interest to examine the distribution of 
ICT literacy for Year 6 and Year 10 students separately and thus investigate the 
extent to which the assessment was appropriately targeted. Figure 3.3 records 
the distributions for Year 6 and Year 10 separately but in the same diagram. 

The data in Figure 3.3 indicate that although the ICT literacy scores of Year 
6 and Year 10 students overlap there is a good level of separation. The mean 
score for Year 10 students was much greater than that of Year 6 students. An 
average Year 10 student was approximately 1.4 times as likely as an average 
Year 6 student to complete an assessment task of average difficulty. 



Reporting ICT Literacy Scale Scores 

In keeping with the practice of other sample surveys in the Australian National 
Assessment Program, the results for ICT literacy have been standardised to 
have a mean score of 400 and a standard deviation of 100 for Year 6 students. 
The choice of these values means that about two-thirds of the Year 6 students 
have ICT literacy scores between 300 and 500 points. It follows from setting 
these scale points that for Year 10 the mean ICT literacy score is 550 and the 
standard deviation is 97.5. 

The analyses provided information about two other properties of the ICT 
literacy scale. The first was that the items formed one dimension. In other 
words the range of items represented one underlying construct. The second 
was that it was reliable in the sense of being internally consistent. In technical 
terms the person separation index was 0.93 (on a metric where 0 would be 
totally unreliable and 1 would be perfectly reliable). 



Establishing and Reporting Proficiency 
Levels 

Although scale scores provide one succinct way of reporting ICT literacy 
overall and for comparisons of different groups of students, it is also possible 
to provide a profile of students’ ICT literacy in terms of proficiency levels. In 
this case six proficiency levels were defined and descriptions were developed 
to characterise typical student performance at each level. The percentage 
of students in each proficiency level could then be calculated. The levels 
and the percentage in each level are used to summarise the performance of 

10 Expressed in more technical terms the difference in the mean for Year 6 and Year 10 was 1.6 
logits which means that the logarithm of the odds ratio was 1.6. 
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students overall, to compare performances across subgroups of students, and 
to compare average performances among groups of students. The proficiency 
levels are set out in Table 3.1. 



Table 3.1: ICT Literacy Proficiency Level Descriptions 



Level I Proficiency level description 



6 Students working at level 6 
create information products 
that show evidence of technical 
proficiency, and careful planning 
and review. They use software 
features to organise information 
and to synthesise and represent 
data as integrated complete 
information products. They 
design information products 
consistent with the conventions 
of specific communication modes 
and audiences and use available 
software features to enhance the 
communicative effect of their work. 



5 Students working at level 5 evaluate 
the credibility of information from 
electronic sources and select the 
most relevant information to use for 
a specific communicative purpose. 
They create information products 
that show evidence of planning and 
technical competence. They use 
software features to reshape and 
present information graphically 
consistent with presentation 
conventions. They design 
information products that combine 
different elements and accurately 
represent their source data. They 
use available software features to 
enhance the appearance of their 
information products. 



4 Students working at level 4 generate 
well targeted searches for electronic 
information sources and select 
relevant information from within 
sources to meet a specific purpose. 
They create information products 
with simple linear structures and 
use software commands to edit 
and reformat information products 
in ways that demonstrate some 
consideration of audience and 
communicative purpose. They 
recognise situations in which ICT 
misuse may occur and explain how 
specific protocols can prevent this. 



I Examples of student achievement at this level 



• create an information product in which 
the flow of information is clear, logical and 
integrated to make the product unified and 
complete. 

• select appropriate key points and data 
from available resources and use their own 
words to include and explicate them in an 
information product. 

• use graphics and text software editing 
features such as font formats, colour, 
animations and page transitions, in ways that 
enhance the structure and communicative 
purpose of an information product. 

• include relevant tables and charts to enhance 
an information product and support these 
representations of data with text that clearly 
explains their purpose and contents. 

• create an information product in which the 
information flow is clear and logical and the 
tone and style are consistent and appropriate 
to a specified audience. 

• select and include information from 
electronic resources in an information 
product to suit an explicit communicative 
purpose. 

• use graphics and text software editing 
features such as font formats, colour and 
animations consistently within an information 
product to suit a specified audience. 

• create tables and charts that accurately 
represent data and include them in an 
information product with text that refers to 
their contents. 

• apply specialised software and file 
management functions such as using the 
history function on a web-browser to return 
to a previously visited page or sorting data 
in a spreadsheet according to a specified 
criterion. 

• create an information product in which the 
flow of information is clear and the tone is 
controlled to suit a specified audience. 

• generate searches that target relevant 
resources and then select relevant sections 
of these resources to include, with some 
modification and supporting text, in an 
information product. 

• apply graphics and text software editing 
features such as, font formats, colour and 
image placement consistently across a simple 
information product. 

• apply infrequently used software and file 
management functions such as displaying a 
specified hidden toolbar in a word processor, 
or using a single pull-down menu function to 
save all the attachments of an email to a new 
location. 



• identify security risks associated with 
internet data and explain the importance of 
respecting and protecting the intellectual 
property rights of authors. 
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Level 


Proficiency level description 


Examples of student achievement at this level 


3 


Students working at level 3 generate 
simple general search questions 
and select the best information 
source to meet a specific purpose. 
They retrieve information from 
given electronic sources to answer 
specific, concrete questions. 

They assemble information in a 
provided simple linear order to 
create information products. They 
use conventionally recognised 
software commands to edit and 
reformat information products. 

They recognise common examples 
in which ICT misuse may occur and 
suggest ways of avoiding them. 


• create an information product that follows a 
prescribed explicit structure. 

• select clear, simple, relevant information from 
given information sources and include it in an 
information product. 

• use graphics and text software editing 
features to manipulate aspects such as 
colour, image size and placement in simple 
information products, 

• apply common software and file management 
functions such as left aligning selected text, 
rotating an image or creating and naming a 
new file on the desktop. 

• recognise the potential for ICT misuse 
such as plagiarism, computer viruses, and 
deliberate identity concealment and suggest 
measures to protect against them. 


2 


Students working at level 2 locate 
simple, explicit information 
from within a given electronic 
source. They add content to and 
make simple changes to existing 
information products when 
instructed. They edit information 
products to create products that 
show limited consistency of design 
and information management. They 
recognise and identify basic ICT 
electronic security and health and 
safety usage issues and practices. 


• locate explicit relevant information or links to 
information from within a web-page. 

• make changes to some presentation elements 
in an information product. 

• apply simple software and file management 
functions such as. copying and pasting 
information from one column of a 
spreadsheet to another column or adding a 
web-page to a list of favourites (bookmarks) 
in a web-browser. 

• recognise common computer use 
conventions and practices such as the use 
of the ‘.edu’ suffix in the URL of a school’s 
website, the need to keep virus protection 
software up-to-date and the need to maintain 
good posture when using a computer. 


1 


Students working at level 1 perform 
basic tasks using computers and 
software. They implement the most 
commonly used file management 
and software commands when 
instructed. They recognise the most 
commonly used ICT terminology and 
functions. 


• apply graphics manipulation software 
features such as adding and moving 
predefined shapes to reproduce the basic 
attributes of a simple image. 

• apply basic file and computer management 
functions such as opening and dragging-and- 
dropping files on the desktop. 

• apply generic software commands such as 
the ‘save as’ and ‘paste’ function or selecting 
all the text on a page. 

• recognise basic computer use conventions 
such as identifying the main parts of a 
computer and that the ‘shut-down’ command 
is a safe way to turn off a computer. 



To form the proficiency levels, the continuum of increasing ICT literacy is 
divided into six levels of equal width (i.e. an equal range of student ability/ 
item difficulty on the scale) with the bottom and top levels being unbounded 
at each extreme. Information about the items in each level has been used to 
develop summary descriptions of the ICT literacy associated with different 
levels of proficiency. These summary descriptions are then used to encapsulate 
ICT literacy of students associated with each level. As a set, the descriptions 
encapsulate a representation of growth in ICT literacy. Table 3.1 describes the 
levels of proficiency in detail. 

The proficiency levels defined in Table 3.1 require a number of decisions so 
that they can be used to summarise and report student performance. The scale 
of ‘ICT literacy’ is continuous and the use of performance levels, or levels of 
proficiency, involves an essentially artificial division of that continuous scale 
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into discrete parts. The number of divisions and the location of the cut-points 
that mark the boundaries of the divisions need to be determined. 

The creation of performance levels involves assigning a range of values on the 
continuous scale to a single level. A procedure similar to that used in the PISA 
study was adopted (OECD, 2004). Students were assigned to the highest level 
for which they would be expected successfully to complete the majority of 
assessment items. If items were spread uniformly across a level, a student near 
the bottom of the level would be expected successfully to complete at least 
half of the assessment items from that level. Students at progressively higher 
points in that level would be expected to correctly answer progressively more 
of the questions in that level. 

The relationship between students and items recognises that there is some 
uncertainty about whether a student could successfully complete any given 
item on the scale (it is based on probabilities). However, it is possible to 
estimate the probability that a student at a particular location on the scale 
(and therefore a particular level) would be expected successfully to complete 
specified items. When the expectation that a student would be able to 
successfully complete ‘at least half of the items’ in a level, the student would 
be placed in that level. 



Illustrative Examples of ICT Proficiency 

The content focus across the levels in the ICT literacy proficiency scale 
described in Table 3.1 shifts and broadens from the lower to the higher levels. 
The lower levels of the scale focus on students’ ICT skills whereas the higher 
levels reflect students’ increasing capacity to use ICT skill to source and reframe 
information for specific communicative purposes. Achievement at the higher 
levels of the scale is demonstrated by students’ sets of responses across HAMs 
that involve research and analysis of information leading up to the production 
of a communicative task. In order for students to provide evidence of research 
and communication using ICT at the higher levels, two of the HAMs that allow 
students to demonstrate these higher levels of achievement are described and 
illustrated in Tables 3.2 and 3.3. These are followed by illustrative examples 
from the two HAMs of student achievement for the higher levels (Levels 6, 5 
and 4) on the scale. 
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Table 3.2: Video Games and Violence Hybrid Assessment Module - Overviews and Large Tasks 



Overview 

Students are told that they are organising a class forum around the topic “Violent Video Games 
and Teenagers”. The students complete a closed information search about the topic, appraise 
and review the quality and trustworthiness of the sourced material and then complete some 
charts of empirical data in preparation for their final communicative task. 



Large Task 

Students prepare a PowerPoint presentation about the relationship between video games and 
violence using a uniform set of the information that they have been working with in the lead-up 
to the large task. The student presentations were assessed against 7 discrete criteria relating 
to the students’ use of the available information and software features with respect to the 
communicative purpose of the presentation. 

Below are screenshots of the four software resources that students used to complete the 
large task. 



Screen 2: Spreadsheet containing data and a 

Screen 1 : New PowerPoint presentation file on chart of the same data that students could make 
which students completed the presentation. use of to complete their presentation. 




Screen 3: Document containing text and data 
from a web-page that students could make use 
of to complete their presentation. 



Screen 4: Document containing text from an 
alternate web-page that students could make 
use of to complete their presentation. 
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Table 3.3: Conservation Project Hybrid Assessment Module - Overview and Large Tasks 



Overview 

Students are told that they are to assist in the planning of a tree-planting conservation project at 
their school. The students complete a closed set of communication and data management tasks 
(such as using email and spreadsheet software) as they collect information in preparation for 
their final communicative task. 



Large Task 

Students add substantive content to and edit a set of broad headings in an electronic document 
in order to produce a report (including recommendations) to their principal about the tree- 
planting project. Students source their information from a closed web environment and data 
in a provided spreadsheet. The student reports were assessed against 4 criteria relating 
to the students’ use of the available information and software features with respect to the 
communicative purpose of the report. 

Below are screenshots of the four software resources that students used to complete the 
large task. 



Screen 1: Report template with headings on 
which students completed the report. 



Screen 2: Homepage of plant information 
website that contains information students 
could make use of to complete the report. 




Screen 3: One of four plant information pages 
within the plant information website that 
students could make use of to complete the 
report. 
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Screen 4: Workbook containing two 
spreadsheets of data that students could make 
use of to complete the report. 
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Illustrative Examples: Levels 6, 5 and 4 



Level 6 

The PowerPoint presentations completed by students working at Level 6 

typically had the following features”: 

• The sequence of slide content was organised with an introduction leading 
through information and argument to a logical conclusion or open ended 
discussion point. 

• The key points from the available resources were included using at least 
some original expression. The key points were linked and explicated with 
students’ own words. 

• The use of colour supported readers’ understanding of the slides by 
showing strong contrast between text and the background and consistency 
of colour use across different text structures (such as headings). 

• The layout of the slides was balanced and the layout features were 
used consistently to fit their communicative purpose throughout the 
presentation. 

• When included, tables and charts were supported by text that clearly 
described their content and meaning. 

The conservation project reports completed by students working at level 6 

typically had the following features: 

• The structure of the report was clear and logical and transitions between 
the sections were used to make the report a single integrated piece. 

• Software tools (such as styles and font types) were used consistently to 
format the components of the report to highlight the structure and support 
understanding of the contribution of the components to the report. 

• Data included in the report had a clear purpose, description and explanation 
and were used to support recommendations for action. 

Level 5 

The PowerPoint presentations completed by students working at Level 5 had 

the following features: 

• The use of colour showed evidence of planning, the use of colour mostly 
highlighted text structures (such as headings). 

• The layout of the slides was largely balanced and the layout features were 
mostly used to fit their communicative purpose. 

• When included, tables and charts were supported by some text to describe 
their content and meaning. 

• The sources of information used in the presentation were cited. 



1 1 The illustrations of work at any given level assume that the features of work at lower levels 
have also been demonstrated or exceeded. This is assumed for all levels. 
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The conservation project reports completed by students working at level 5 
typically had the following features: 

• The structure of the report was clear and logical. 

• The tone and style of the report were consistent throughout the report and 
the report was clearly targeted to the principal (audience). 

• Data were included in the report with some explanation. Recommendations 
for action were included. 

Students working at Level 5 were also able to complete information analysis 
tasks such as describing three potential problems when downloading files from 
the internet and identifying different reasons that suggest the information on a 
website might be unreliable. 

Level 4 

The PowerPoint presentations completed by students working at Level 4 had 
the following features: 

• There was some organisation in the sequence of the slide content and 
these were supported by the use of headings. 

• Relevant charts and tables were copied and pasted into the presentation. 

• Relevant pieces of text (usually sentences) were copied from resources and 
pasted into the presentation. Some of these sentences were semantically 
linked with student’s own words. 

The conservation project reports completed by students working at level 4 
typically had the following features: 

• The report style was largely consistent and showed evidence of being 
targeted to the principal as the audience. 

• Information from a range of the available software sources was included in 
the report. 

Students working at Level 4 were also able to complete information management 
and analysis tasks such as searching for a file with the word ‘greenhouse’ in it, 
selecting the most reliable website from a summary set returned by a search 
engine, or explaining why some software is created with an expiry date. 
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Illustrative Examples: Levels 3, 2 and 1 

As the ICT literacy scale extends downwards from Level 6, the proportion 
of scale content detailing skills and simple, single process information 
management (such as editing or adding text for example) increases and the 
proportion of scale content detailing students’ creation of original ICT content 
decreases. As such, the illustrations of achievement at these lower levels tend 
to be student responses to discrete tasks, rather than global judgements that 
can be made across large pieces of student work (such as the presentation and 
report that were used to illustrate achievements at Levels 6, 5 and 4). Following 
are examples of assessment items that are indicative of achievement at each of 
Levels 3, 2 and 1. Three items, one from each strand in the ICT literacy progress 
map, have been selected as indicative of achievement at each level. 

Level 3 

Level 3 Illustrative Example 1 




in this example students were asked whether it was ‘okay’ to use text produced 
by another person in their own work. Students working at Level 3 were 
typically able to identify either that using the work of another person without 
permission or citation was plagiarism or that in order to use the work of others 
it is necessary to seek permission from the author or, if only a small extract was 
to be used, to cite the source of the extract. This item was aimed to broadly 
measure student understandings of the basic principles of appropriation and 
citation of work rather than exploring the detailed (legal) nuance of copyright 
and permissions. This item is considered to represent Strand C of the ICT 
literacy progress map. 
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Level 3 Illustrative Example 2 




In this example students are asked to change the colour of information in a 
table (both changing the font colour and changing the shading of the cells in 
the table were acceptable responses) and also to change the name of one of the 
inactive worksheets in the given (Microsoft Excel) workbook. Students working 
at Level 3 were typically able to make both the requested amendments to the 
workbook. This item is considered to represent Strand B of the ICT literacy 
progress map. 



Level 3 Illustrative Example 3 




In this example students were asked to use the search engine provided to 
search for information on the topic “Playing violent video games makes 
teenagers more likely to be violent themselves”. Students working at Level 
3 were typically able to include three of the following five key categories of 
search term in their information search: 
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1. violent/violence 

2. video 

3. games 

4. teenagers/adolescents 

5. cause/influence/effect (or a synonym) 

This item is considered to represent Strand A of the ICT literacy progress 
map. 

Level 2 

Level 2 Illustrative Example 1 




In this example students were asked to provide advice about how to alleviate 
neck soreness resulting from computer use. Students working at Level 2 were 
typically able to provide any single piece of simple computer-use advice such 
as adjusting the chair or screen height or using stretching exercises and 
rest breaks. This item is considered to represent Strand C of the ICT literacy 
progress map. 
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Level 2 Illustrative Example 2 



Flag Design 
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In this example students are asked to read an email and click on the hyperlink 
in the email. Students working at Level 2 were typically able to click on the 
hyperlink in the email. In completing this item successfully, students needed 
to know the term ‘hyperlink’ and also how to use the mouse to click on a 
hyperlink. This item is considered to represent Strand B of the ICT literacy 
progress map. 

Level 2 Illustrative Example 3 




In this example students were asked to click on (i.e. locate and click on) the 
link on a webpage that will help them locate information about symbols of the 
Northern Territory. Students working at Level 2 were typically able to locate 
and click on the ‘Territory symbols’ link. This item is considered to represent 
Strand A of the ICT literacy progress map. 
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Level 1 

Level 1 Illustrative Example 1 



General Skills 
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In this example students were asked identify the best way to turn off a 
computer from a set of four multiple-choice options. Students operating at 
Level 1 typically could identify selecting ‘Shut Down’ as the best way to turn 
off a computer. This item was successfully completed by more students than 
any other item in the assessment. The item is considered to represent Strand 
C of the ICT literacy progress map. 

Level 1 Illustrative Example 2 




Task Overview 



Example Item 



In this example students were asked to use the simple graphics software 
shown in the Example Item (above right) to create a replica of the flag seen in 
the Task Overview (above left). Students working at Level 1 were typically able 
to select the background image by selecting ‘background’ from the menu and 
then selecting the appropriate background from a set (each background was 
shown on the screen when selected). Similarly students working at Level 1 were 
typically also able to select the koala image from a list of images and relocate 
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and resize the image to match the given source. These tasks are considered to 
represent Strand B of the ICT literacy progress map. 

Level 1 Illustrative Example 3 




In this example students were asked to open a single specified file from the 
screen. Students operating at Level 1 typically could open the file. Students 
could use any method to open the file (such as double-clicking with the mouse, 
or using a menu option). The item is considered to represent Strand A of the 
ICT literacy progress map. 



A Profile of ICT Literacy 

On the basis of the student scores on the ICT literacy scale it is possible to 
develop a profile of Australian students in Year 6 and Year 10 in terms of the 
distribution of their levels of ICT literacy. Table 3.4 represents a profile of 
ICT literacy for Year 6 and Year 10 students in Australian schools. It contains 
information about the score range for each proficiency level and the percentage 
of Year 6 and Year 10 students in each proficiency level. Moreover, for each of 
the proficiency levels it is possible to characterise the tasks that a student at 
that level would be expected to complete successfully. 

Only 0.6 per cent of Year 6 students and 0.1 per cent of Year 10 students 
performed at a level below the lower boundary of proficiency level 1. Only one 
Year 1 0 student exceeded the upper bound of the top proficiency level (Level 6). 
Thus the six proficiency levels fully span the range of student performance. 

Figure 3.4 shows the distribution of Year 6 and Year 10 students over the 
proficiency levels characterised by the descriptors in Table 3.4. The data in 
Figure 3.4 indicate the difference between Year 6 and Year 10. Only eight per 
cent of Year 6 students performed at Level 4 or above compared to 61 per cent 
of Year 10 students. In contrast 51 per cent of Year 6 students performed at 
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Level 2 or below compared to seven per cent of Year 10 students. The overlap 
was mostly contained to one proficiency level - Level 3. 

ICT Literacy Proficiency Distribution 



Year 6 
Proficient 
Standard 



^ 20 



Level 1 



Level 2 



Year 10 
Proficient 
Standard 




Level 3 

Proficiency Level 



Level 4 



Level 5+ 



Note: Confidence intervals are shown for the percentages in each level. 

Figure 3.4: Distribution of Year 6 and Year 10 Students over ICT Literacy Proficiency Levels 
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Table 3.4: ICT Literacy Profiles for Year 6 and Year 10 



Year 6 



Year 10 



750 



625 



500 



0% 


Level 6 

Students working at level 6 create information products that show evidence 
of technical proficiency, and careful planning and review. They use software 
features to organise information and to synthesise and represent data as 
integrated complete information products. They design information products 
consistent with the conventions of specific communication modes and 
audiences and use available software features to enhance the communicative 
effect of their work. 


1 0.4% 


0.1% 


Level 5 

Students working at level 5 evaluate the credibility of information from 
electronic sources and select the most relevant information to use for a 
specific communicative purpose. They create information products that show 
evidence of planning and technical competence. They use software features 
to reshape and present information graphically consistent with presentation 
conventions. They design information products that combine different 
elements and accurately represent their source data. They use available 
software features to enhance the appearance of their information products. 


\ 11.9% 


7.7% 


Level 4 

Students working at level 4 generate well targeted searches for electronic 
information sources and select relevant information from within sources to 
meet a specific purpose. They create information products with simple linear 
structures and use software commands to edit and reformat information 
products in ways that demonstrate some consideration of audience and 
communicative purpose. They recognise situations in which ICT misuse may 
occur and explain how specific protocols can prevent this. 


48.9% \ 


/40.8% 


Level 3 

Students working at level 3 generate simple general search questions and 
select the best information source to meet a specific purpose. They retrieve 
information from given electronic sources to answer specific, concrete 
questions. They assemble information in a provided simple linear order to 
create information products. They use conventionally recognised software 
commands to edit and reformat information products. They recognise 
common examples in which ICT misuse may occur and suggest ways of 
avoiding them. 


/ 32.0% 


38.s\ 


Level 2 

Students working at level 2 locate simple, explicit information from within a 
given electronic source. They add content to and make simple changes to 
existing information products when instructed. They edit information 
products to create products that show limited consistency of design and 
information management. They recognise and identify basic ICT electronic 
security and health and safety usage issues and practices. 


1 1 

/ 6.4% 


12.6% \ 


Level 1 

Students working at level 1 perform basic tasks using computers and 
software. They implement the most commonly used file management and 
software commands when instructed. They recognise the most commonly 
used ICT terminology and functions. 


1 0.4% 



lOp 



375 



6p 



250 



Year 6 



Year 10 



6p: Year 6 proficient standard 
lOp: Year 10 proficiient standard 
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Establishing Proficient Standards 

In addition to deriving the ICT Literacy proficiency scale, proficient standards 
were established for each of Year 6 and Year 10. The proficient standards 
represent points on the proficiency scale that represent a ‘challenging but 
reasonable’ expectation for typical Year 6 and 10 students to have reached by 
the end of each of those years of study. The proficient standards are important 
because they provide reference points of reasonable expectation of student 
achievement on the scale, but also because the standards refer to Year 6 
which is the penultimate or ultimate year of primary schooling, and Year 10. 
In some senses the standards can be considered as markers of ICT literacy 
preparedness for students as they begin the transition to next stages of their 
educational or vocational lives. 

The two proficient standards (one for Year 6 and one for Year 10) were 
established as a result of consultations (over two days for each Year level) 
with ICT education experts and representatives from all states and territories 
and all school sectors. The standards-setting groups included currently 
practising teachers with specific ICT expertise, ICT curriculum experts and 
educational assessment experts. The Year 6 and Year 10 proficient standards 
were established on different days. Some members of the standards setting 
group worked on both standards, whilst some participated only in the Year 6 
or Year 10 standards setting exercise. 

In each case, the same process was used to generate the recommendations 
for the position of the proficient standards on the scale. The experts were first 
introduced to the notion of the ‘challenging but reasonable’ standard so that a 
common conceptual understanding of this notion was established. The process 
of establishing the proficiency cut-points used a combined modified-Angoff 
Method (Angoff, 1971) and Bookmark Method (Lewis, Mitzel & Green, 1999) 
for each of Grades 6 and 10. This process resulted in a recommendation from 
each expert for the item on the scale that should be deemed as the minimum 
achievable by a proficient student at Year 6 or Year 10. 

In the modified-Angoff Method the experts were presented with each individual 
assessment item and asked simply to select ‘Yes’ or ‘No’ to identify whether 
they believed that a typical Year 6 or 10 student (depending on which level 
was being established) could reasonably be expected to complete the item 
successfully. These individual expert data were then collated and each expert 
received a summary data sheet that included their own rating for each item, the 
percentage of all expert raters who selected ‘Yes’ and ‘No’ for each item and the 
percentage of students in the target year level who successfully completed the 
item in the national assessment. The experts were then given the opportunity 
to consider and discuss their judgements. 

In the Bookmark Method the experts were provided a list of the full set of 
assessment items in order from least difficult to most difficult according to the 
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percentage of students at the target year level who successfully completed 
each item. The experts were then asked in groups to work their way up from the 
bottom of the scale and select the item that they felt was the most difficult that 
could reasonably be expected of a student in the target year level according to 
the notion of the ‘challenging but reasonable’ standard. Although these item 
cut-points were discussed in groups, each expert was finally asked to select 
the single item that they believed represented the appropriate ‘challenging but 
reasonable’ cut-point for the target grade level on the scale of all items. 

These recommendations provided the basis for defining the cut-point of 
marginal proficiency for each of Year 6 and Year 10. Although there was a range 
of cut-point recommendations among the experts there was no overlap between 
the highest Year 6 recommendation and the lowest Year 10 recommendation. 
For each year level the experts’ expectations of student achievement by item 
was consistently higher than the actual student achievement data suggested. 

The set of expert judgements of the ‘challenging but reasonable’ proficient 
standard items for each Year level was used as the basis for establishing the 
proficient standards as points on the ICT Literacy proficiency scale for each of 
Years 6 and 10. 

• The proficient standard for Year 6 was defined as the boundary between 
levels 2 and 3 or a score of 375 on the ICT literacy scale. From Table 3.4 it 
can be seen that 49 per cent of Year 6 students reached or exceeded the 
Year 6 proficient standard. 

• The proficient standard for Year 10 was defined as the boundary between 
levels 3 and 4 or a score of 500 on the ICT literacy scale and 61 per cent of 
Year 10 students reached or exceeded the Year 10 proficient standard. 



Summary 

Student responses to the items that made up the various modules in the ICT 
literacy assessment were manifestations of a single underlying dimension of 
ICT literacy. Those items formed a scale that ranged from less to greater ICT 
literacy that could be measured reliably. The scale was standardised so that 
the mean score for Year 6 was 400 and the standard deviation for Year 6 was 
100 points. Students from Year 10 recorded higher ICT literacy scores than 
students in Year 6 with the difference being 151 scale points (or 1.5 Year 6 
standard deviations). 

The ICT literacy scale could also be described in terms of six described 
proficiency levels that provide a profile of progress in ICT literacy from 
students at level 1 who “perform basic tasks using computers and software, 
implementing commonly used file management and software commands and 
recognising most commonly used ICT terminology and functions” to students 
at level 6 who “are able to create information products that show evidence 
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of technical proficiency, careful planning and review, use software features to 
organise information, synthesise and represent data as integrated information 
products, design information products consistent with the conventions of 
specific communication modes and audiences and use available software 
features to enhance the communicative effect of their work.” 

Forty-nine per cent of Year 6 students reached or exceeded the Year 6 proficient 
standard by demonstrating the ability to “generate simple general search 
questions and select the best information source to meet a specific purpose, 
retrieve information from given electronic sources to answer specific, concrete 
questions, assemble information in a provided simple linear order to create 
information products, use conventionally recognised software commands to 
edit and reformat information products”. 

Sixty-one per cent of Year 10 students reached or exceeded the Year 10 
proficient standard by demonstrating the ability to “generate well targeted 
searches for electronic information sources and select relevant information 
from within sources to meet a specific purpose, create information products 
with simple linear structures and use software commands to edit and reformat 
information products in ways that demonstrate some consideration of audience 
and communicative purpose.” 
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Chapter 4 

Patterns of ICT Literacy 



Australia’s national goals for schooling assert that schooling should be socially 
just, so that: 

Students’ outcomes from schooling are free from the effects of negative forms 
of discrimination based on sex, language, culture and ethnicity, religion 
or disability; and of differences arising from students’ socio-economic 
background or geographic location (MCEETYA, 1999: Goal 3.1). 

From the accumulated results of studies of student achievement in a wide range 
of fields, it is known that student achievement is influenced by factors such as: 
sex, socioeconomic background, language background, geographic location, 
and Indigenous status. Students come from a wide range of backgrounds and it 
is important to understand the extent to which these factors relate to their iCT 
literacy. ICT is a wide-reaching aspect of life in modern Australian society and 
students who do not develop proficiency in ICT are likely to be limited in the 
extent to which they can participate fully in economic and social life. 

This chapter examines the relationship between students’ iCT literacy and 
their personal and family backgrounds, it focuses on differences in iCT literacy 
between different groups of students in Year 6 and in Year 10. As part of the 
Sample Assessment of ICT literacy students completed a background survey 
through which information was collected about students’ sex, socioeconomic 
background. Indigenous status, language background, school location and age. 
Those data provide the basis for a social mapping of patterns of ICT literacy 
across Australia, in addition, because education is a responsibility of State 
and Territory authorities, the survey data are used to investigate the extent to 
which there are variations in iCT literacy among States and Territories. 
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Three approaches are used to report differences in ICT literacy among groups 
of students. The first is to compare the mean achievement scores of groups 
on the standardised ICT literacy scale. This is the most powerful comparison 
because it makes use of the full distribution of data. The second is to compare 
the percentages of students from each group who are in each of the proficiency 
levels. The third is to compare the percentages of students in each group who 
have attained the proficient standard for the Year level. 



Differences in ICT Literacy among States 
and Territories 

Comparison of means for Year 6 and Year 10 

Table 4.1 records the mean ICT literacy score for each State and Territory 
together with the 95 per cent confidence interval that indicates the level of 
accuracy with which the mean was measured (the narrower the confidence 
interval, the more accurate the measurement)'^. 



Table 4.1: Means and Confidence Intervals for ICT Literacy by State and Territory 





Year 6 


Year 10 


State or Territory 


Mean Score 


Confidence 

Interval 


Mean Score 


Confidence 

Interval 


New South Wales 


404.9 


±12.9 


550.6 


±13.1 


Victoria 


423.5 


±13.7 


565.1 


±9.8 


Queensland 


369.6 


±12.3 


546.6 


±11.6 


South Australia 


411.9 


±11.4 


547.1 


±11.0 


Western Australia 


379.4 


±10.8 


535.3 


±11.8 


Tasmania 


404.2 


±19.4 


538.1 


±11.8 


Northern Territory 


345.8 


±53.7 


515.3 


±28.2 


Australian Capital Territory 


428.4 


±22.1 


571.8 


±17.8 


ALL 


400.0 


±6.3 


550.6 


±5.7 



Notes: (a) Differences in confidence intervals reflect differences in sample sizes for jurisdictions 
as detailed in Table 2.4 as well as differences in the variation within jurisdictions. The larger 
confidence intervals for the Northern Territory and the Australian Capital Territory reflect the 
smaller sample sizes for those jurisdictions. For the Northern Territory the effect of the smaller 
sample size is compounded by the large variation in scores within the jurisdiction. 



12 Each State and Territory mean is an estimate of the total population value, inferred from the 
result obtained by the sample of students tested. Because it was an estimate, it was subject to 
uncertainty. If the mean scores were estimated from different samples drawn from the same 
population of students, the actual results for the mean would vary a little. However, one can 
be confident that the population mean lies between the value obtained and about two standard 
errors (actually 1.96) on either side of it. This range is the confidence interval. According to 
statistical theory, the estimate of the mean from repeated sampling would be expected to fall 
within the range for 95 of 100 samples drawn. 
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The mean score for any State or Territory can be compared with that of any 
other State or Territory so that if the confidence intervals do not overlap 
the difference in the means is statistically significant. The differences in ICT 
literacy among States and Territories are greater at Year 6 than at Year 10. The 
difference between the highest jurisdictional mean ICT literacy score for Year 6 
and the lowest is 83 scale points. At Year 10 the difference between the highest 
and lowest mean scores is 56 scale points. Correspondingly the differences 
between the second highest and second lowest mean scores for jurisdictions 
are 54 and 30 scale points for Year 6 and Year 10 respectively. 

The data in Table 4.1 also indicate the difference in the mean scores between 
Year 6 and Year 10 which can be taken as an indication of the change or growth in 
ICT literacy between Year 6 and Year 10. For Australia as a whole this difference 
was 151 scale points. There was some variation among jurisdictions in the 
difference between Year 6 and Year 10 with the largest being for Queensland 
(177 points) and the smallest being for South Australia and Tasmania (134 to 
135 points). Patterns are illustrated in Figure 4.1. 



700 1 
650 - 
600 - 
550 - 
500 - 
450 - 
400 - 
350 - 
300 - 
250 - 
200 - 




NSW VIC 
Y6 YIO Y6 YIO 




QLD SA WA TAS NT 
Y6 YIO Y6 YIO Y6 YIO Y6 YIO Y6 YIO 




ACT AUS 
Y6 YIO Y6 YIO 



+ 1 standard 
deviation 



mean & 
confidence 
interval 

-1 standard 
deviation 



Note: Confidence intervals are shown for each jurisdiction 
Figure 4.1: Mean ICT Literacy Scores by Year Level and Jurisdiction 



Multiple comparison of jurisdictional means for Year 6 
and Year 10 

Table 4.2 records which of the multiple comparisons of Year 6 jurisdictional 
means are statistically significant at the five per cent level. These pair-wise 
comparisons are shown in lower left-hand quadrant of Table 4.2 as a series of 
symbols to indicate whether the difference for the comparison is statistically 
significant or not. Those comparisons that are statistically significant are 
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indicated by the upward- or downward-pointing symbols and those that are 
not are indicated by the dot. From these data it can be seen that the mean 
scores for the Australian Capital Territory Victoria and South Australia are 
significantly higher than the means for Western Australia, Queensland and the 
Northern Territory but not significantly different from the means for New South 
Wales and Tasmania. The mean for New South Wales is significantly higher 
than that for Western Australia and Queensland and the mean for Tasmania is 
significantly higher than that for Queensland. 



Table 4.2: Multiple Comparisons of Mean Year 6 ICT Literacy by State and Territory 





Mean 


Conf. 

Inter. 


ACT 


VIC 


SA 


NSW 


TAS 


WA 


QLD 


NT 


Australian 

Capital 

Territory 


428.4 


±22.1 




• 


• 


• 


• 


A 


A 


A 


Victoria 


423.5 


±13.7 






□ 1 




• 


A 


A 


A 


South 

Australia 


411.9 


±11.4 


• 


• 




• 


• 


A 


A 




New South 
Wales 


404.9 


±12.9 


• 


• 


l9 






A 


A 




Tasmania 


404.2 


±19.4 


• 


• 


• 


• 




• 


A 




Western 

Australia 


379.4 


±10.8 


▼ 


▼ 


▼ 


▼ 


:a 


■ 


e: 




Queensland 


369.6 


±12.3 


▼ 


▼ 


▼ 


▼ 


▼ 


• 






Northern 

Territory 


345.8 


±53.7 


▼ 


▼ 


▼ 


• 


• 


• 


PI 


■ 



Notes: 

Read across the row to compare one jurisdictions mean with other jurisdictions. The source and 
comparison jurisdictions are listed as the row and column headings respectively. 

Results in the lower left-hand quadrant do not include the Bonferroni adjustment. Results in the 
upper right-hand quadrant incorporate the Bonferroni adjustment. 

Legend: • denotes no significant difference in mean scale scores 

▼ denotes mean is significantly lower than the comparison jurisdiction 
A denotes mean is significantly higher than the comparison jurisdiction 



However, there is an argument that, when making multiple comparisons (that is, 
comparing the performance of one jurisdiction with those of the others in the 
set), an allowance needs to be made for the possibility that a comparison could 
appear significant by chance (since one is making many comparisons from the 
same data). Multiple comparison significance tests that limit the probability of 
mistakenly finding a difference in performance were applied and the results are 
recorded in the top right-hand quadrant of Table 4.2. This adjustment is called 
the Bonferroni adjustment'^ In practice applying the Bonferroni adjustment 
made little difference to the pattern simply rendering the comparison between 
South Australia and the Northern Territory non-significant. 



13 When and whether the Bonferroni adjustment should be applied is the subject of ongoing 
debates among statisticians. Those debates centre on whether the comparisons are simple 
pair-wise comparisons or are genuinely multiple comparisons. The Bonferroni adjustment was 
applied by the OECD to country comparisons in PISA in 2000. In PISA 2003 both adjusted and 
un-adjusted results were reported and in PISA 2006 the adjustment will not be applied. 
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Table 4.3 records the multiple comparisons of the Year 10 means for ICT literacy. 
As was recorded in the previous table comparisons without the Bonferroni 
adjustment are recorded in the lower left-hand quadrant and comparisons 
incorporating the Bonferroni adjustment are recorded in the upper right-hand 
quadrant. Applying the Bonferroni adjustment made no difference to the 
results for Year 10 students, it can be seen that fewer of these comparisons 
are statistically significant than for the Year 6 comparisons in Table 4.2. The 
mean ICT literacy scores for the Australian Capital Territory and Victoria 
are significantly greater than those for Tasmania, Western Australia and the 
Northern territory. No other means are significantly different at the five per 
cent level. 



Table 4.3: Multiple Comparisons of Mean Year 10 ICT Literacy by State and Territory 





Mean 


Conf. 

Inter. 


ACT 


VIC 


NSW 


SA 


QLD 


TAS 


WA 


NT 


Australian 

Capital 

Territory 


571.8 


±17.8 




• 


• 


• 


• 


A 


A 


A 


Victoria 


1 565.1 1 


±9.8 


13 


m\ 


nn 


• 


• 


A 


A 


A 


New South 
Wales 


550.6 


±13.1 


• 


• 




• 


• 


• 


• 


• 


South 

Australia 


547.1 


±11.0 


• 


• 


• 




• 


• 


• 


• 


Queensland 


546.6 


±11.6 


• 


• 


• 


• 




• 


• 


• 


Tasmania 


538.1 


±11.8 


▼ 


▼ 


• 


• 






Eira 


Western 

Australia 


535.3 


±11.8 


▼ 


▼ 


• 


• 


• 


• 




• 


Northern 

Territory 


515.3 


±28.2 


▼ 


▼ 


• 


• 


• 


• 


Q 


■ 



Notes: 

Read across the row to compare one jurisdictions mean with other jurisdictions. The source and 
comparison jurisdictions are listed as the row and column headings respectively. 

Results in the lower left-hand quadrant do not include the Bonferroni adjustment. Results in the 
upper right-hand quadrant incorporate the Bonferroni adjustment. 

Legend: • denotes no significant difference in mean scale scores 

▼ denotes mean is significantly lower than the comparison jurisdiction 
▲ denotes mean is significantly higher than the comparison jurisdiction 



Comparisons of jurisdictional means with the national 
mean 

Another approach to the examination of jurisdictional means is to make 
comparisons with the national mean using the data recorded in Table 4.1. 
When this comparison is made at Year 6 level it appears that there are three 
groups of jurisdictions. 

• Year 6 students from Victoria and the Australian Capital Territory have 
means that are statistically significantly greater than the national mean. 

• Year 6 students from Queensland and Western Australia are statistically 
significantly lower than the national mean. 
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• Year 6 students from New South Wales, South Australia, Tasmania and the 
Northern Territory are not significantly different from the national mean 
(although for the Northern Territory this is largely a result of the large 
confidence interval that reflects the small sample size). 

In the case of Year 10 students only the Northern Territory mean was 
significantly different from the national mean. 

This comparison of jurisdictional means does tend to mask differences because 
the jurisdictions are part of the overall comparison group. However, it is a 
comparison that is of general interest and is therefore reported here. 



Comparison of distributions 

In addition to examining the mean ICT literacy scores, the distributions of 
student achievement scores for States and Territories were examined. Figure 

4.2 displays the scaled means and distributions for States and Territories at 
Year 6 and Year 10. 

in each case the length of the bar shows the range of scores for the middle 80 
per cent of students (those between the 10th and 90th percentiles)'^. The outer 
shaded section shows the range of one standard deviation above and below 
the mean. The centre darker section is the confidence interval associated with 
the mean. The multiple comparisons of the means reported in Tables 4.2 and 

4.3 correspond to the overlap, or lack of overlap, between the darker shaded 
areas for jurisdictions on the vertical achievement scale. 

The spread of scores achieved by the middle 80 per cent of Year 6 students 
across all students in Australia was approximately 243. The Northern Territory 
had the widest spread of scores (a range of about 257 scale points). South 
Australia had the smallest spread of scores (232 scale points). 

The spread of scores achieved by the middle 80 per cent of Year 10 students 
was 233 scale points. Again the Northern Territory had the widest spread of 
scores (249 scale points). Queensland had the lowest spread of scores (217 
scale points). 



14 The 10th percentile is the point on the scale below which 10% of the student scores are located 
and the 90th percentile is the point on the scale above which 10% of the student scores are 
located. 
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Year 6: ICT Literacy Score Distributions 
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Year 10: ICT Literacy Score Distributions 
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Fignre 4.2: Distributions of ICT Literacy Scores in States and Territories 
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Table 4.4: Percentages of Year 6 Students at Each Proficiency Level on the ICT Literacy Scale by 
State and Territory 





Percentage in Each Proficiency Level 


% at or 




Level 1 


Level 2 


Level 3 


Level 4 


Above 4 


above 

Proficient 

Standard 


New South Wales 


10.5 


39.1 


41.8 


8.6 


0.1 


50.5 


(confidence interval) 


±3.3 


±5.2 


±6.0 


±3.6 


±0.2 


±6.6 


Victoria 


8.6 


33.6 


47.4 


10.3 


0.1 


57.9 


(confidence interval) 


_ ±3.8 


±4.7 


±4.5 


±3.4 


±0.1 1 


±6.3 


Queensland 


19.3 


43.0 


33.6 


4.0 


0.1 


37.7 


(confidence interval) 


±4.8 


±4.7 


±4.8 


±1.7 


±0.3 


±5.3 


South Australia 


10.4 


37.8 


42.7 


8.7 


0.3 


51.7 


(confidence interval) 


±3.6 


±5.7 


±4.0 


±3.6 


±0.7 


±5.0 


Western Australia 


17.2 


43.1 


35.0 


4.4 


0.2 


39.6 


(confidence interval) 


±4.7 


±4.9 


±5.3 


±1.9 


±0.8 


±5.4 


Tasmania 


10.3 


40.8 


40.4 


8.0 


0.4 


48.9 


(confidence interval) 


±5.1 


±7.7 


±8.4 


±4.5 


±1.1 


±9.0 


Northern Territory 


24.2 


39.7 


33.3 


2.8 


0.0 


36.0 


(confidence interval) 


±12.2 


±11.5 


±9.0 


±2.6 


±0.0 


±10.0 


Australian Capital 
Territory 


8.5 


33.1 


45.5 


12.8 


0.1 


58.4 


(confidence interval) 


±4.9 


±11.4 


±9.9 


±7.0 


±0.8 


±12.5 


Australia 


12.6 


38.8 


40.8 


7.7 


0.1 


48.6 


(confidence interval) 


±1.6 


±2.3 


±2.7 


±1.5 


±0.1 


±3.0 



Comparison of percentages in proficiency levels 

Table 4.4 shows the percentage of Year 6 students who attained each of the 
proficiency levels across the States and Territories. 

Overall, 49 per cent of students attained Level 3 or above (the Year 6 proficient 
standard). In an analogous manner to the multiple comparisons of jurisdictional 
means it is also possible to compare the percentages of students attaining 
the proficient standard for Year 6 and Year 10. Table 4.5 records which of the 
comparisons of percentage of Year 6 students attaining the proficient standard 
are statistically significant at the five per cent level. These are shown as a series 
of symbols to indicate whether the difference for the comparison is statistically 
significant or not. Those comparisons that are statistically significant are 
indicated by the upward- or downward-pointing symbols and those that are 
not are indicated by the dot. 

Table 4.5 records comparisons of the percentage attaining the proficient 
standard in each jurisdiction with the percentage of students across Australia 
that attained the proficient standard. Victorian students performed significantly 
above the Australian average of 49 per cent*'’. There was no significant 



15 Even though the boundaries of the confidence intervals touch a more precise analysis by 
testing the z-score of the difference indicates that the difference is statistically significant. 
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difference between the national performance and that of students in Tasmania, 
New South Wales and South Australia. In Queensland and Western Australia the 
proportion of students attaining the proficiency level was significantly lower 
than the Australian average. 



Table 4.5: Multiple Comparisons of Percentage of Year 6 Students Attaining the Proficient 
Standard in ICT Literacy by State and Territory 




Australian 

Capital 

Territory 



Victoria 



South 

Australia 



New South 
Wales 



Tasmania 



Western 

Australia 



Queensland 



Northern 

Territory 



Australia 



Notes: 

The Bonferroni adjustment has not been applied to these data because it was developed for 
multiple comparisons of means rather than percentages. 

Read across the row to compare one jurisdictions mean with other jurisdictions. The source and 
comparison jurisdictions are listed as the row and column headings respectively. 

Legend: • denotes no significant difference in percentages 

▼ denotes percentage is significantly lower than the comparison jurisdiction 
A denotes percentage is significantly higher than the comparison jurisdiction 



For the Australian Capital Territory and the Northern Territory the sample 
sizes limit our certainty about the differences from the national average even 
though they are relatively large in magnitude. The percentage of students 
attaining the proficient standard in the Australian Capital Territory was not 
significantly different from the national average (although the difference was 
nine percentage points). For the Northern Territory the percentage of students 
attaining the proficient standard was not significantly different from the national 
average (although the gap was 13 percentage points). 

However, when the comparisons among jurisdictions including confidence 
intervals are taken into account, the groups appear to be slightly different 
with overlap between the groups. It remains clear that Western Australia and 
Queensland perform lower than most of the other jurisdictions. Victorian 
students performed significantly above Western Australia and Queensland 
but did not differ significantly from South Australia, New South Wales and 
Tasmania. 
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Table 4.6 records the percentage of Year 10 students attaining each proficiency 
level in each jurisdiction. Table 4.7 provides information regarding the 
percentage of Year 10 students attaining the proficient standard for each 
jurisdiction and in comparison with the national percentage. For Australia 
overall, 61 per cent attained Level 4 or above (the proficient standard). There 
were no significant differences in the percentage of students attaining the 
proficient standard across the States and Territories for Year 10. 



Table 4.6: Percentages of Year 10 Students at Each Proficiency Level on the ICT Literacy Scale by 
State and Territory 





F 


’ercentage i 


n Each Proficiency Leve 


1 


% at or 




Level 2 
or below 


Level 3 


Level 4 


Level 5 


Level 6 


above 

Proficient 

Standard 


New South Wales 


7.1 


31.8 


49.4 


11.2 


0.5 


61.1 


(confidence interval) 


±2.5 


±7.5 


±6.4 


±3.3 


±1.0 


±7.6 


Victoria 


5.9 


27.6 


49.1 


16.7 


0.7 


66.5 


(confidence interval) 


±1.9 


±4.5 


±5.0 


±4.1 


±1.2 J 


±4.8 


Queensland 


5.7 


34.8 


49.0 


10.4 


0.2 


59.5 


(confidence interval) 


±2.8 


±6.9 


±8.1 


±3.1 


±0.5 


±7.4 


South Australia 


6.1 


32.5 


49.4 


11.6 


0.4 


61.4 


(confidence interval) 


±2.4 


±4.1 J 


±5.3 


( ±3.6 


L*0.6 j 


±5.0 


Western Australia 


9.3 


34.9 


47.6 


8.2 


0.0 


55.8 


(confidence interval) 


±4.2 


±4.7 


±5.6 


±3.0 


±0.2 


±6.1 


Tasmania 


8.7 n 


35.0 


47.2 


9.0 


ro.i 


56.4 


(confidence interval) 


±4.2 1 


±7.0 


J 


±3.9 


±0.4 


±6.4 


Northern Territory 


14.4 


37.0 


40.9 


7.7 


0.0 


48.6 


(confidence interval) 


±11.3 


±8.1 


±13.6 


±5.9 


±0.0 


±13.2 


Australian Capital 
Territory 


4.0 


30.5 


47.5 


17.5 


0.5 


65.5 


' (confidence interval) 


±3.1 


±12.5 


±7.4 


±8.7 


±1.3 


±11.4 


Australia 


6.8 


32.0 


48.9 


11.9 


0.4 


61.2 


(confidence interval) 


±1.2 


±2.9 


±2.7 


±1.5 


±0.4 


±3.1 



58 







Table 4.7: Multiple Comparisons of Percentage of Year 10 Students Attaining the Proficient 
Standard in ICT Literacy by State and Territory 




Victoria 



Australian 

Capital 

Territory 



South 

Australia 



New South 
Wales 



Queensland 



Tasmania 



Western 

Australia 



Northern 

Territory 



Notes: 

Read across the row to compare one jurisdictions mean with other jurisdictions. The source and 
comparison jurisdictions are listed as the row and column headings respectively. 

Legend: • denotes no significant difference in percentages 

▼ denotes percentage is significantly lower than the comparison jurisdiction 
A denotes percentage is significantly higher than the comparison jurisdiction 



Summary 

There were differences in the means and dispersion of student ICT literacy 
among States and Territories at Year 6. For Year 6 the gap in ICT literacy between 
the top two jurisdictions (the Australian Capital Territory and Victoria) and the 
bottom two jurisdictions (Queensland and the Northern Territory) was a little 
more then 50 scale points. The difference between the top two and bottom 
two jurisdictions in the percentages of students attaining the Year 6 proficient 
standard was approximately 20 percentage points. 

For Year 10 students the differences among jurisdictions were not as great. 
The gap in mean ICT literacy scores between the Australian Capital Territory 
and the Northern Territory was 56 scale points but the gap between Victoria 
and Western Australia (the next widest gap) was 30 scale points. There were 
no significant differences among jurisdictions in the percentage of students 
attaining the proficient standard at Year 10. The range covering the six States 
was 11 percentage points with a gap between Victoria and the Northern 
Territory of 18 percentage points. 
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Differences in ICT Literacy between Males 
and Females 



It was anticipated that there may have been differences in ICT literacy between 
male and female students. While there was a 14 scale point difference in mean 
scores in favour of female students for Year 6 and nine scale points at Year 10, 
these differences were not statistically significant. Relevant data are recorded 
in Table 4.8. 



Table 4.8: Differences in ICT Literacy between Male and Female Students at Year 6 and Year 10 



State 


Year 6 


Year 10 


Males 


Females 


Males 


Females 


Mean 

Score 


Conf. 

Interval 


Mean 

Score 


Conf. 

Interval 


Mean 

Score 


Conf. 

Interval 


Mean 

Score 


Conf. 

Interval 


NSW 


399.2 


±19.6 


411.6 


±12.3 


549.0 


±15.7 


552.5 


±15.5 


VIC 


419.5 


±18.4 


427.5 


±12.7 


561.8 


±11.6 


568.5 


±16.7 


QLD 


355.9 


±16.5 


382.9 


±14.7 


538.8 


±17.2 


554.2 


±9.6 


SA 


400.1 


±16.3 


421.7 


±14.3 1 


539.1 


±15.5 


554.2 


±15.5 


WA 


375.3 


±13.5 


383.9 


±13.3 


526.9 


±15.3 


542.9 


±12.0 


TAS 


402.7 


±16.9 


406.1 


±25.1 


534.1 


±18.8 


543.0 


±16.9 


NT 


334.8 


±52.1 


362.9 


±56.3 


514.1 


±30.0 


516.7 


±40.8 


ACT 


415.1 


±27.6 


437.9 


±29.8 


568.1 


±29.0 


575.2 


±21.0 


ALL 


392.9 


±9.2 


407.4 


±6.5 


546.2 


±7.6 


555.4 


±6.9 



Table 4.9 records the percentages of male and female students in each 
proficiency level and those at or above the proficient standard for the relevant 
Year level. At Year 6 there are seven percentage points more females than 
males who had attained the proficient standard but this difference was not 
statistically significant (i.e. the confidence intervals overlap). At Year 10 there 
were three percentage points more females than males who had attained the 
proficient standard and this difference was also not statistically significant. A 
greater proportion of females than males attain the proficient standard for ICT 
but the difference is small. 
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Table 4.9: Percentages of Year 6 and Year 10 Students at Each Proficiency Level on the ICT 
Literacy Scale by Sex 



Year 6 Year 10 





Males 


Females 


Males 


Females 


Per 

cent 


Conf. 

Interval 


Per 

cent 


Conf. 

Interval 


Per 

cent 


Conf. 

Interval 


Per 

cent 


Conf. 

Interval 


Level 1* 


14.4 


±2.6 


10.7 


±2.0 


8.1 


±1.8 


5.5 


±1.7 


Level 2* 


40.3 


±4.2 


37.3 


±3.8 


Level 3 


37.5 


±4.0 


44.2 


±4.3 


32.3 


±3.9 


31.6 


±3.6 


Level 4 


7.7 


±1.9 


7.7 


±2.0 


48.0 


±3.6 


49.7 


±3.4 


Level 5** 


0.2 


±0.3 


0.1 


±0.2 


11.2 


±2.3 


12.8 


±2.2 


Level 6** 


0.4 


±0.6 


0.4 


±0.5 


Proficient 

Standard 


45.4 


±4.9 


52.0 


±4.1 


59.6 


±4.2 


62.9 


±3.5 



* The percentages of Year 10 students at Levels 1 and 2 have been combined because the 
individual percentages within each level are very low. 

** The percentages of Year 6 students at Levels 5 and 6 have been combined because the 
individual percentages within each are very low. 



Differences in ICT Literacy by 
Socioeconomic Group 

Parental occupation was used as the indicator of socioeconomic group. Data 
based on parental education have not been reported because of the high 
levels of respondents who indicated that they did not know their parents 
education (33% of Year 6 and 13% of Year 10). The occupations of parents were 
provided by students and classified into five categories following the PMRT 
classification: (1), senior managers and professionals; (2), other managers and 
associate professionals; (3), tradespeople and skilled office, sales and service 
staff; (4), unskilled labourers, office, sales and service staff; and (5), not in paid 
work in the last 12 months. 

Where occupations were available for two parents, the higher coded occupation 
was used as the indicator of socioeconomic group. Mean scores for each group 
of students (based on the parental occupation that was the higher in cases 
where two parental occupations were indicated) are recorded in Table 4.10. 
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Table 4.10: Differences in ICT Literacy among Socioeconomic Groups at Year 6 and Year 10 





Mean 


Confidence 


Number of 


Group 


Score 


Interval 


Cases 



Year 6 Students 

Senior managers and professionals 
Other managers and associate professionals 
Skilled trades, clerical and sales 
Unskilled manual, office and sales 
All coded students 



Year 10 Students 


Senior managers and professionals 


586.2 


±9.4 


599 


Other managers and associate professionals 


560.3 


±7.0 


1,327 


Skilled trades, clerical and sales 


542.4 


±6.6 


958 


Unskilled manual, office and sales 


520.6 


±10.8 


545 


All coded students 


553.8 


±5.8 


3,429 



Note: Table does not include students who indicated that their parents had not been in paid 
work for 12 months. There were 107 students in Year 6 who had a mean ICT literacy score of 
332.6 and 65 students at Year 10 who had a mean ICT literacy score of 476.3. The numbers in this 
group are too small to generate estimates with any precision. 



450.3 


±11.7 


505 


424.4 


±6.0 


1,097 


392.3 


±7.9 


1,016 


363.1 


±8.5 


873 


403.3 


±6.1 


3,491 



The data in Table 4.10 show that the differences among socioeconomic groups 
are significant and substantial. At both Year 6 and Year 10 the differences 
between each group and the adjacent group are statistically significant. For 
Year 6 students the mean ICT literacy score of those students whose parents 
were in occupations classified as “senior managers and professionals” was 87 
points higher than for those whose parents were in occupations classified as 
“unskilled manual, office and sales”. For Year 10 students the corresponding 
gap was 65 points. Students whose parents had not been in paid employment 
for 12 months scored at or below the level for those whose parents worked in 
occupations classified as “unskilled manual, office and sales” but the numbers 
were too small to provide stable estimates of performance. 

Notwithstanding the level of missing data, it was possible to compare the mean 
scores of Year 10 students whose parents had completed Year 12 with those 
whose parents had not completed Year 12. The mean ICT literacy score for 
Year 10 students whose parents had completed Year 12 was 562 scale points 
and for those whose parents had not completed Year 12 the mean score was 
529 scale points; a difference of 33 scale points. Among Year 6 students the 
corresponding values for the mean ICT literacy scores were 420 scale points 
for those whose parents had completed Year 12 and 390 scale points for those 
whose parents had not completed Year 12. These results provide an indication 
of the difference in ICT literacy associated with parental school attainment but 
the estimates should be treated with caution because of the level of data that 
could not be coded, especially among Year 6 students. 
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Table 4.11: Percentages of Year 6 and Year 10 Students at Each Proficiency Level on the ICT 
Literacy Scale by Socioeconomic Group 





Unskilled manual, 
office & sales 


Skilled trades, 
clerical & sales 


Other managers 
&ass. 

professionals 


Senior managers 
& professionals 




Per 

cent 


Coni. 

Interval 


Per 

cent 


Conf. 

Interval 


Per 

cent 


Conf. 

Interval 


Per 

cent 


Conf. 

Interval 


Year 6 Students 


Level 1 


19.0 


±3.3 


13.0 


±2.9 


7.1 


±2.2 


4.5 


±2.6 


Level 2 


48.9 


±6.2 


40.9 


±4.4 


34.1 


±4.8 


27.4 


±5.6 


Level 3 


28.9 


±5.5 


40.3 


±4.5 


49.0 


±5.4 


52.3 


±7.0 


Level 4 


3.1 


±1.9 


5.8 


±2.2 


9.8 


±2.6 


14.9 


±5.2 


Level 5 


0.1 


±0.2 


0.0 


±0.0 


0.1 


±0.2 


0.7 


±1.0 


Level 6 


Proficient 

Standard 


32.0 


±5.7 


46.1 


±4.5 


58.9 


±4.8 


68.0 


±6.1 



Year 10 Students 





Level 2 


11.2 


±4.6 


6.9 


±2.3 


5.8 


±1.6 


2.3 


±1.9 


Level 3 


40.0 


±6.6 


36.0 


±5.9 


28.5 


±3.9 


22.3 


±5.3 


Level 4 


41.7 


±7.7 


46.8 


±6.0 


52.2 


±4.2 


55.2 


±6.0 


Level 5 


6.9 


±3.8 


9.8 


±2.9 


13.0 


±2.7 


19.8 


±4.2 


Level 6 


0.2 


±0.6 


0.5 


±0.9 


0.5 


±0.6 


0.4 


±0.8 


Proficient 

Standard 


48.8 


±7.0 


57.1 


±5.8 


65.7 


±4.0 


75.4 


±5.4 



Note: In Year 6 29.5 per cent of the 107 students whose parents had been unemployed for 12 
months attained the proficient standard in ICT literacy. In Year 10 32.0 per cent of the 65 students 
whose parents had been unemployed for 12 months attained the proficient standard in ICT 
literacy. In both cases the confidence intervals are large (+/- 14 and +/- 22 percentage points) 



Table 4.11 records the corresponding data as the percentage of students in 
each proficiency level by socioeconomic group. Those data indicate that twice 
the percentage of Year 6 students from the “senior manager and professional” 
group (approximately two thirds) as from the “unskilled manual, office and 
sales” group (approximately one third) attained the proficient level in ICT 
literacy. Among Year 10 students the gap between these two socioeconomic 
groups was just a little smaller, in Year 10, 75 per cent more students from the 
“senior manager and professional” group (three-quarters) compared with 49 
per cent from the “unskilled manual, office and sales” group (approximately 
half) attained the proficient level in ICT literacy. 

From the data in Tables 4.10 and 4.11 it is clear that there is a substantial 
association between socioeconomic background and ICT literacy that is similar 
to that in other fields (Sirin, 2005). 
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Table 4.12: Differences in ICT Literacy between Indigenous and Non-Indigenous Students at 
Year 6 and Year 10 




Differences in ICT Literacy by Indigenous 
Status 

Indigenous Year 6 and Year 10 students’ mean ICT literacy relative to that of 
non-Indigenous students is shown in Table 4.12. At both year levels, Indigenous 
students did not perform as well as non-Indigenous students on the ICT literacy 
Scale. The gap between the non-Indigenous and Indigenous students was about 
70 scale points at both year levels. This difference is statistically significant 
and substantial. It is similar to the differences reported between Indigenous 
and non-Indigenous students in other studies of achievement. 

The percentage of Indigenous and non-Indigenous students at each proficiency 
level are shown in Table 4.13. It can be seen that the distribution of Indigenous 
students across the proficiency levels is skewed towards the lower levels 
compared to the distribution of non-Indigenous students. At Year 6, 25 per cent 
of Indigenous students were located in proficiency level 1 compared to 1 1 per 
cent of non-Indigenous students. In contrast, only one per cent of Indigenous 
students in Year 6 were located in proficiency level 4 and above compared to 
eight per cent of non-Indigenous students. 

There is a similar pattern in Year 10. Approximately 21 per cent of Indigenous 
students were located in proficiency levels 1 and 2 compared to six per cent 
of non-Indigenous students. Six per cent of Indigenous students were located 
in proficiency levels 5 and 6 compared to 13 per cent of non-Indigenous 
students. 

The pattern of ICT literacy can be summarised by the observation that the 
percentages of non-Indigenous students attaining the proficient standard 
for each Year were substantially greater than the percentages of Indigenous 
students. In Year 6 the comparison is 50 per cent compared to 30 per cent. In 
Year 10 the comparison is 62 per cent compared to 35 per cent. 
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Table 4.13: Percentages of Year 6 and Year 10 Students at Each Proficiency Level on the ICT 
Literacy Scale by Indigenous Status 



Year 6 Year 10 





Indigenous 


Non-Indigenous 


Indigenous 


Non-Indigenous 




Per 

cent 


Conf. 

Interval 


Per 

cent 


Conf. 

Interval 


Per 

cent 


Conf. 

Interval 


Per 

cent 


Conf. 

Interval 


Level 1 


25.2 


±10.6 


11.4 


±1.6 


20.7 


±10.1 


6.4 


±1.1 


Level 2 


44.9 


±15.6 


38.5 


±2.5 


Level 3 


28.7 


±13.1 


41.9 


±2.9 


44.3 


±12.7 


31.3 


±2.9 


Level 4 


1.1 


±3.1 


8.1 


±1.6 


29.2 


±10.9 


49.6 


±2.8 


Level 5 


fi 1 


A 


n 1 


9 


5.8 


±5.8 


12.3 


±1.6 


Level 6 


U. 1 


±U.4 


U. 1 


±u.z 


0.0 


±0.0 


0.4 


±0.4 


Proficient 

Standard 


29.9 


±12.9 


50.1 


±3.1 


35.0 


±11.5 


62.3 


±3.1 



Differences in ICT Literacy by Language 
Background 

Table 4.14 compares the mean scores of students who spoke languages other 
than English at home with students who spoke only English. There was no 
significant difference between these groups of students at either year level. 
At Year 10 students with a language background other than English scored 
slightly lower than students who spoke only English at home but the difference 
was not statistically significant. The distributions across the proficiency levels 
of students who spoke languages other than English at home compared with 
those students who spoke only English are shown in Table 4. 1 5. A similar pattern 
to that shown by the means is evident. At both year levels, the proportion 
of students who speak languages other than English at home achieving the 
proficient standard was not different from the proportion of those who speak 
only English at home. 



Table 4.14: Differences in ICT Literacy between Students with a Language Background Other 
than English and Other Students at Year 6 and Year 10 



Language 

background 


Mean Score 


Confidence 

Interval 


Number of 
Cases 


Year 6 








Other than English 


399.8 


±12.3 


850 


English 


399.9 


±6.0 


2,859 


All 


399.9 


±6.1 


3,709 


Year 10 


Other than English 


544.8 


±11.2 


810 


English 


552.8 


±5.9 


2,769 


All 


550.7 


±5.8 


3,579 
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Table 4.15: Percentages of Year 6 and Year 10 Students at Each Proficiency Level on the ICT 
Literacy Scale by Language Background 





Year 6 


Year 10 




Language 
Background 
Other than 
English 


English-speaking 

Background 


Language 
Background 
Other than 
English 


English-speaking 

Background 




Per 

cent 


Conf. 

Interval 


Per 

cent 


Conf. 

Interval 


Per 

cent 


Conf. 

Interval 


Per 

cent 


Conf. 

Interval 


Level 1 


13.5 


±3.7 


12.2 


±1.9 








±1.5 


Level 2 


37.7 


±4.9 


' 39.4 


±2.7 




±Z. < 


D.D 


Level 3 


40.2 


±5.4 


41.0 


±3.2 


33.4 


±5.1 


31.1 


±3.0 


Level 4 


8.7 


±2.6 


1 


±1.6 


45.9 


±5.9 


50.0 


±3.0 


Level 5 


0.0 ±0.0 


0.2 


±0.2 


12.2 


±3.4 


12.0 


±1.7 


Level 6 


0.6 


±1.1 


0.4 


±0.4 


Proficient 

Standard 


48.8 


±6.2 


48.5 


±3.2 

1 


58.6 


±5.6 


62.3 

1 


±3.3 



Differences in ICT Literacy by Geographic 
Location 

Table 4.16 shows the mean scores on the ICT literacy scale of students living 
in metropolitan, provincial and remote areas. At both Year 6 and Year 10 the 
tendency was for metropolitan students to record higher ICT literacy scores 
than did students in provincial areas who, in turn recorded higher scores than 
those in remote areas. 



Table 4.16: Differences in ICT Literacy among Students from Metropolitan, Provincial and 
Remote Locations at Year 6 and Year 10 



Geographic Location 


Mean Score 


Confidence 

Interval 


Number of 
Cases 


Year 6 








Metropolitan 


408.2 


±8.2 


2,402 


Provincial 


385.9 


±9.7 


1,153 


Remote 


344.9 


±47.9 


121 


All 


400.5 


±6.2 


3,676 


Year 10 


Metropolitan 


554.5 


±7.3 


2,345 


Provincial 


544.8 


±12.0 


1,069 


Remote 


504.4 


±23.2 


132 


1 All 


551.0 


±5.8 


3,546 



Among Year 6 students, the difference between metropolitan and provincial 
students was statistically significant as was the difference between 
metropolitan and remote students. Among Year 10 students the difference 
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between metropolitan and remote students, and the difference between 
provincial and remote students, was statistically significant. The fact that 
the difference between metropolitan and provincial locations is replicated in 
the two independent samples (Year 6 and Year 10) suggests confidence in the 
proposition that the difference between these locations is a real difference and 
not something that appeared by chance. 

Table 4.17 records the percentages of students at each proficiency level from 
each the three locations; metropolitan, provincial and remote. The pattern is 
similar to that shown by the differences in mean ICT literacy scores and is 
replicated in Year 6 and Year 10 patterns. Students from metropolitan locations 
tend to be located towards the higher proficiency levels to a greater extent 
than their peers in provincial or remote locations. The pattern is apparent in 
the percentages of students attaining the proficient standard. 



Table 4.17: Percentages of Year 6 and Year 10 Students at Each Proficiency Level on the ICT 
Literacy Scale by Geographic Location 





Metropolitan 


Provincial 


Remote 




Per cent 


Conf. 

Interval 


Per cent 


Conf. 

Interval 


Per cent 


Conf. 

Interval 


Year 6 Students 


Level 1 


11.2 


±2.0 


14.2 


±3.2 


26.1 


±15.1 


Level 2 


36.8 


±3.1 


43.2 


±5.4 


41.4 


±15.7 


Level 3 


42.9 


±3.1 


37.4 


±5.8 


30.1 


±18.1 


Level 4 


8.8 


±1.9 


5.2 


±2.4 


2.5 


±4.2 


Level 5 
Level 6 


0.2 


±0.2 


0.1 


±0.2 


0.0 


±0.0 


Proficient 

Standard 


51.9 


±3.8 


42.7 


±5.5 


32.6 


±18.9 


I Year 10 Students 


Level 1 


6.6 


±1.5 


7.2 


±2.9 


15.4 


±8.0 


Level 2 


Level 3 


30.6 


±3.8 


34.2 


±5.4 


38.8 


±9.7 


Level 4 


49.2 


±3.4 


48.5 


±5.4 


39.0 


±10.8 


Level 5 


13.2 


±2.3 


9.7 


±3.7 


6.7 


±4.8 


Level 6 


0.4 


±0.4 


0.4 


±0.9 


0.1 


±0.6 


Proficient 

Standard 


62.8 


±4.1 


58.6 


±5.7 


45.8 


±9.7 



in Year 6, 52 per cent of students from a metropolitan location attained the 
proficient standard compared with 43 per cent of students from a provincial 
location and 33 per cent of students from a remote location who attained 
the proficient standard. The differences between metropolitan and remote 
locations were statistically significant and the difference between metropolitan 
and provincial locations was just on the boundary of statistical significance. 
The difference between provincial and remote locations was large but it is not 
statistically significant because of the large confidence interval associated 
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with the estimate based on the relatively small number of students in remote 
locations. 

In Year 10, 63 per cent of students from a metropolitan location attained the 
proficient standard compared with 59 per cent of students from a provincial 
location and 46 per cent of students from a remote location who attained the 
proficient standard. The difference between metropolitan and remote locations 
was statistically significant but that between metropolitan and provincial 
locations was not statistically significant. 

Overall, it can be concluded that geographic location is associated with ICT 
literacy. The very large confidence interval of the remote students (associated 
with the small numbers in remote locations) results in some differences not 
being statistically significant. However, the certainty that these differences 
are real differences is supported by the fact that they are replicated in both 
samples. 



Net Influences on ICT Literacy: Results of a 
Regression Analysis 

The net influence of student characteristics on ICT literacy was examined using 
multiple regression analysis. This provides an indication of the effect of each 
influence on ICT literacy after allowing for the effects of associated variables. 

A regression analysis is based on an equation that has ICT literacy as the 
outcome and the other variables as predictors. The analysis generates 
coefficients that provide an indication of the net influences of the predictor or 
independent variables in the analysis (e.g. parental occupation status) on the 
dependent variable (ICT literacy). The larger the coefficient is, the stronger 
the effect of that variable as a predictor on the dependent variable. Results 
of the regression analyses for students in Year 6 and students in Year 10 are 
shown in parallel in Table 4.18. 

In Table 4.18 the magnitude of the regression coefficient (B) represents the 
size of the net effect of each predicator on the ICT literacy scale units (where 
the mean for Year 6 is 400 and the standard deviation is 100 units). For a 
continuous variable the magnitude of the regression coefficient represents the 
net effect of a one unit difference in the predictor on the ICT literacy score. 
For a dichotomously coded variable (e.g. sex) the magnitude of the coefficient 
is the net effect of the difference between having that characteristic and not 
having that characteristic on the performance measure. 

Table 4.18 indicates the percentage of the variance explained by the groups of 
independent variables on performance. It indicates how much of the variation 
in student scores can be accounted for by the combination of variables that 
have been included in the analysis to that stage. Two overall observations can 
be made from these data. The first is that the largest source of variation among 
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those variables included was parental occupational group. The second is that 
most of the variation in students’ ICT literacy is not accounted for by these 
variables representing student characteristics. 

Table 4.18 also shows the confidence intervals associated with the regression 
coefficients. If the magnitude of the coefficient is greater than the confidence 
interval it can be inferred that the coefficient is significantly different from zero 
and that there an effect of that predictor on ICT literacy that has not arisen as 
a result of chance'*^. 

The analysis of influences on performance was conducted by entering blocks 
of variables in sequence. Of course at the final stage of the process the result 
is the same as if all variables had been analysed simultaneously. However, the 
block-wise process provides additional information. Firstly, the results at each 
stage indicate how much the model is improved by including additional blocks 
of variables. Secondly it is possible to examine changes in the regression 
coefficients as additional blocks are added and thus infer the extent to which 
the observed effects are direct or transmitted. 

• Block 1 included age and sex. 

• Block 2 included Indigenous status (Indigenous or not Indigenous) and 
language background other than English. 

• Block 3 contained the variables concerned with parental occupation. 
Because parental occupation was coded in one of four groups it was 
represented as a set of dummy variables (coded as 0 or 1 to reflect 
whether the parental occupation was in that group). These were senior 
managers and professionals, other managers and associate professionals, 
tradespeople and skilled office, sales and service staff. The reference 
category was unskilled labourers, office, sales and service staff and the 
results for the other occupational groups are relative to that group. 

• Block 4 is home location represented as a set of dummy variables (coded as 
0 or 1 to reflect whether the student came from a metropolitan, provincial 
or remote area). Metropolitan location was the reference category and the 
results reported are relative to students in a metropolitan location. 



16 The confidence intervals are based on replication methods (specifically the Jack-knife method) 
so that they take account of the clustered sample structure. With the complex sample designs 
that are multi-level but also involve explicit and implicit stratification, differential sampling 
fractions between strata, probability proportional to size selection and other complexities 
empirical replication methods provide better estimates than multi-level modelling methods. 
Multi-level modelling assumes simple random samples from an infinite population at each level 
which is not the way samples for national assessment surveys are selected. PISA and TIMSS 
use replication methods to estimate precision because they also involve complex sample 
designs. In these analyses all the variables are student level except for state and that is simply 
a category - there are no school level variables in the analyses. 

17 When categorical variables involving more than two categories are included in a regression 
analysis it is necessary to designate one category as the reference category and exclude 
that from the analysis. As a consequence the results that are obtained are relative to the 
reference category. It is conventional to choose as a reference category one which contains 
sufficient cases to have a relatively low confidence interval and one which is near either end 
of the distribution to facilitate interpretation. In these analyses the reference category for 
socioeconomic group was unskilled labourers, office, sales and service staff; for location 
the reference category was “metropolitan” and for jurisdiction the reference category was 
Queensland. 
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• Block 5 is State or Territory represented as a set of dummy variables 
(coded as 0 or 1 for each state). Queensland is the reference jurisdiction 
(that was necessarily excluded from the analysis) and the results reported 
are relative to students in Queensland. 



Table 4.18: Results of Regression Analysis of ICT Literacy on Student Characteristics 





Year 6 Students 


Year 10 Students | 








Variance 






Variance 


Predictor 


Regression 

Coefficient 

(B) 


Conf. 

Interval 


Explained 

(R 

squared) 


Regression 

Coefficient 


Conf. 

Interval 


Explained 

(R 

squared) 


Intercept 

bO 


451.4 






1135.7 






Block 1 

Age (years) 


-7.2 


±11.2 




-37.0 


±10.6 




Sex (female 1 , 
male 0) 


11.2 


±8.2 


01 


6.1 


±8.0 


01 


Block 2 














Indigenous 
(coded 1.0) 


-35.6 


±21.0 




-48.7 


±21.0 




Language other 
than English 
(coded 1.0) 


-8.1 


±12.0 


04 


-7.8 


±11.0 


03 


Block 3 (compared to students with parents in unskilled occupations) 




Senior 
managers & 
professionals 


78.1 


±12.3 




61.5 


±12.2 




Managers 
& associate 


52.2 


±9.8 


11 


36.5 


±11.0 


07 


professionals 














Tradespeople, 
skilled office, 
sales service 


24.8 


±9.0 




20.1 


±10.6 




Block 4 (compared to metropolitan students) 








Provincial 

location 


-14.3 


±12.2 


12 


-3.7 


±13.7 


07 


Remote location 


-27.3 


±23.5 




-30.8 


±25.1 




Block 5 (compared to Queensland students) 








New South 
Wales 


33.1 


±14.7 




23.9 


±17.2 




Victoria 


50.4 


±16.7 




37.9 


±14.9 




South Australia 


34.9 


±14.7 




19.1 


±14.1 




Western 

Australia 

Tasmania 


5.4 

43.7 


±11.8 

±20.0 


15 


-11.6 

24.6 


±14.1 

±16.5 


10 


Northern 

Territory 

Australian 


5.9 


±33.1 




-0.5 


±30.4 




Capital 

Territory 


44.2 


±25.1 




38.5 


±22.1 




Full model 






14.7% 






9.6% 



* Regression coefficients in bold are significant (a< .05). 

Results of the regression analysis are shown in Table 4.18'*. The student 
characteristics that had the greatest influence on iCT literacy were 
socioeconomic group and Indigenous status. Students whose parents were in 
the “senior manager and professional occupational” group had ICT literacy 



18 In Table 4.18 the regression coefficients are those from the full model whereas the percentage 
variance is reported for each block of variables. Greater detail regarding the effects of each 
step in the block-wise analysis on the regression coefficient will be reported elsewhere. 
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scores between 78 (Year 6) and 62 (Year 10) scale points higher than those 
whose parents were in the “unskilled labourers, office, sales and service staff” 
group (the reference category). The differences above the reference category 
for children of the “other managers and associate professionals” group were 
52 (Year 6) and 37 (Year 10) points and for children of the “tradespeople 
and skilled office, sales and service staff” the difference above the reference 
category were 25 (Year 6) and 20 (Year 10) points. 

Indigenous students had ICT literacy scores that were lower than that of non- 
Indigenous students by 36 scale points at Year 6 and 49 scale points at Year 10. 
Although these differences are moderate they are smaller than the differences 
that are observed in the national assessment of Civics and Citizenship and in 
successive cycles of PISA and TIMSS (the comparison is possible because in all 
of those studies the reporting scales have a similar standard deviation) 

There was a significant net effect of living in a remote location compared to 
living in a metropolitan location at both Year levels. Students from remote 
locations had lower ICT literacy scores than metropolitan students at Year 6 (27 
points) and Year 10 (31 points). Students from provincial locations had lower 
ICT literacy scores than their peers in metropolitan location for Year 6 (14 
points) but there was no net difference between provincial and metropolitan 
locations in Year 10. 

The effect of student sex was only evident in Year 6 (females had higher ICT 
literacy scores by 11 points) and age was only evident at Year 10. in Year 10 
younger students had higher ICT literacy scores by three points per month 
compare to older students. Table 4.18 also indicates the net effects for each 
State or Territory after allowance is made for the effects of differences in social 
and demographic characteristics. The data recorded in Table 4.18 indicate the 
net difference between the listed State and Queensland which was chosen as 
the reference). 

On the basis of these analyses it could be concluded that Western Australia, 
Queensland and the Northern Territory have comparatively low ICT literacy 
scores in Year 6 and Year 10, net of any effects of differences in social and 
demographic characteristics. The other states have relatively higher ICT 
literacy scores at both Year 6 and Year 10. The process of adjusting also 
reduced the extent to which the Northern Territory fell behind that of other 
jurisdictions. 



19 It is possible to compare this difference with those reported in other National Assessment 
Program and with results from PISA and TIMSS because in all of those studies the reporting 
scale is based on a standard deviation of 100 scale points. Indeed other scales could be 
transformed to a common standard deviation so as to enable comparison of effect size in 
each. By way of comparison the difference between Indigenous and non-Indigenous students 
in Civics and Citizenship was 70 scale points, in the 2003 science literacy assessment at Year 6 
the difference was 66 points, in PISA 2003 and PISA 2000 the difference in mathematical literacy 
the difference was 86 scale points, in PISA 2003 the difference in reading literacy was 83 scale 
points and in TIMSS at Year 8 the difference was 68 scale points. The overall conclusion is that 
the differences between Indigenous and non-Indigenous students in ICT literacy are a little 
smaller than in these other area. 
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Victoria and the Australian Capital Territory are the highest scoring jurisdictions 
at both Year 6 and Year 10 (and it should be noted that this result is net of the 
influence of differences in the social composition of the student population in 
these jurisdictions). The net advantage to these two jurisdictions averaged 
just less than 50 points at Year 6 and approximately 38 points at Year 10. 

New South Wales and South Australia also performed better than the 
reference group but by approximately 34 points at Year 6 and 22 points at 
Year 10. Tasmanian Year 6 students performed at a similarly high level to their 
counterparts in the Australian Capital Territory in Year 6 but more like those 
in New South Wales and South Australia in Year 10. 

It is of interest that the net effects associated with the jurisdictions are 
consistent between Year 6 and Year 10 (the net effect is consistently lower 
at Year 10 than Year 6 but patterns of differences between the jurisdictions 
remain fairly consistent across the Year levels). This suggests that there may 
be influences associated with the provision of ICT in schools that contributes 
to those differences. 

The data in Table 4.18 also indicate that the combination of these social 
and demographic characteristics accounts for little of the variation in ICT 
literacy: 15 per cent in Year 6 and ten per cent in Year 10. Possibly there are 
differences in student experience of using ICT not included in this analysis 
that are associated with social and demographic characteristics. These will be 
investigated further in Chapter 5. 

Another perspective on the results of the analysis derives from the amount of 
variance explained as each block of variables is added to the analysis. Student 
age and sex account for very little of the variance and those variables in 
combination with Indigenous status and language background account for only 
3 or 4 per cent of the variance in students scores. Inclusion of socioeconomic 
background (block 3) adds an additional 7 per cent to the explained variance in 
the case of Year 6 and 4 per cent in the case of Year 1 0. Inclusion of location adds 
very little to the explained variance but the inclusion of the block of variables 
representing jurisdiction adds a further 3 per cent to the variance. Overall, the 
variables account for 15 per cent of the variance in Year 6 student ICT literacy 
scores and 10 per cent of the variance in Year 10 ICT literacy scores. 
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Concluding Comments 

Student background characteristics are related to ICT literacy and the patterns 
are similar in Year 6 and Year 10. The largest effects contributing to differences 
of 60 to 70 scale points are associated with socioeconomic background. The 
analyses cannot indicate whether this is associated with differences in access 
to and opportunity to use ICT or other factors but it indicates potentially 
something that might influence later opportunities. Indigenous status is also 
associated with ICT literacy to a moderate extent (after allowance is made for 
the associated influence of socioeconomic group and geographic location). 

There was evidence of disadvantage in the development of ICT literacy for 
students from remote locations. The effect was consistently observed in Year 
6 and Year 10. Whether those differences are associated with access and 
opportunity will be explored in the next chapter. 

Finally, the analyses indicated that there were differences among jurisdictions 
that could not be accounted for by differences in social and demographic 
characteristics. The extent to which those differences can be attributed to 
differences in curriculum and teaching remains a topic for further study. 
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Chapter 5 

Familiarity with ICT 



In the international context, Australian students have substantial familiarity 
with ICT at home and school. Data from the Programme for International 
Student Assessment (PISA) conducted in 2003 indicate that Australia has one of 
the highest levels of computer availability in secondary schools among OECD 
countries, with an average of 3.3 students per computer compared to an OECD 
average of 6.3 students per computer (OECD, 2005). These findings also point 
to an improvement in school computing resources in Australia over the three 
years since 2000 when there was an average of 4.5 students per computer. 

Data from PISA 2003 also provide information about the extent to which 15- 
year-old students have access to computers at home with 93 per cent of 
Australian 15-year-old students indicating that they had a computer at home 
which they could use for school work. The OECD average was 79 per cent and 
the Australian level of access was exceeded only by the Netherlands, Korea, 
Sweden and Norway and was greater than the level of access in the United 
Kingdom and the United States. PISA 2003 also revealed that 96 per cent of 
Australian 15-year-old students had a computer of some type at home and 83 
per cent had a link to the internet. 

Findings from the Trends in International Mathematics and Science Study 
(TIMSS) suggest similar high levels of access at home and school among primary 
school students in 2002. Ninety-two per cent of Australian Year 4 students had 
a computer at home and 81 per cent of those students use a computer both at 
school and at home (Martin et al., 2004). The percentage of Australian Year 4 
students indicating that they have a computer at home is comparable to the 
percentage in the United States, England and the Netherlands. 
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Despite these overall levels of access it is important to map the variations in 
familiarity with ICT because there is some evidence of differences in access 
across socioeconomic groups and small differences between metropolitan and 
non-metropolitan locations. The data gathered as part of the national sample 
study provide the opportunity to provide this mapping for Year 6 and Year 
10 students (using the same survey questions), to investigate the ways in 
which students use ICT and to investigate the links between ICT literacy and 
familiarity with ICT 



Student Familiarity with ICT 

In the national assessment survey two aspects of familiarity with ICT were 
investigated. The first was the length of time for which students had been 
using computers and the second was the frequency with which students used 
computers. 

Student experience of using computers 

Table 5.1 records the extent of time students in Year 6 and Year 10 had been 
using computers. From the data in Table 5.1 it is evident that most school 
students have three or more years experience of using computers. Eighty per 
cent of Year 6 students and 86 per cent of Year 10 students had been using 
computers for three years or more. 



Table 5.1: Student Experience of Computer Use 



Year 6 
(N = 3746) 
Year 10 

(N = 3627) 




Note: Confidence intervals are shown in parentheses 



Frequency of computer use 

The national survey data also indicate that students were frequent users of 
computer technology. Relevant data are recorded in Table 5.2. Those data 
indicate that computer use is more frequent at home than at school for both 
Year 6 and Year 10 students. Forty-three per cent of Year 6 students and 58 per 
cent of Year 10 students use computers at home every day. In comparison 14 
per cent of Year 6 students and 18 per cent of Year 10 students use a computer 
at school every day. Computer use at home is more frequent among Year 10 
students than Year 6 students but there is no significant difference in the 
frequency of computer use at school between Year 6 and Year 10 students. 
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Table 5.2: Frequency of Computer Use at Home and School for Year 6 and Year 10 Students 





Frequency of computer use 


Mean days 
per month 


Never 


Less than 
monthly 


Weekly to 
monthly 


Few times 
per week 


Every day 


Computer use at home 


Year 6 


5.7 


3.8 


10.3 


37.1 


43.1 


12.6 




(±1.3) 


(±0.8) 


(±1.0) 


(±2.2) 


(±2.7) 


(±0.4) 


Year 10 


3.6 


2.6 


8.1 


27.3 


58.4 


14.6 




(±0.9) 


(±0.7) 


(±1.4) 


(±1.9) 


(±2.5) 


(±0.4) 


Computer use at school 


Year 6 


0.9 


3,5 


22.9 


58.6 


14.0 


9.2 




(±0.5) 


(±0.9) 


(±2.2) 


(±3.6) 


(±2.8) 


(±0.4) 


Year 10 


2.1 


7.5 


25.3 


47.1 


18.1 


8.9 




(±1.4) 


(±1.3) 


(±2.3) 


(±2.7) 


(±2.4) 


(±0.4) 



Notes: Confidence intervals (95%) are shown in parentheses 

Mean days per month calculated on the basis of 20 working days per month and five working 
days per week: Never = 0; less than monthly = 1; weekly to monthly = 4; few times per week = 10; 
every day = 20. 

Variations in experience of using computers 



In order to investigate differences among groups of students in their experience 
of using computers the percentage of students who had used a computer for 
more than five years was used as an indicator. 

Among Year 6 and Year 10 students there were no significant differences 
between the percentages of males and females, or the percentages of students 
from different geographic locations, who had been using a computer for more 
than five years. 

There were just a few significant differences among States and Territories in 
experience of using computers. Details are recorded in Table 5.3. Among Year 
6 students, experience of computer use in South Australia was greater than 
that in New South Wales, Queensland, Western Australia, Tasmania and the 
Northern Territory by a statistically significant amount. In addition experience 
of using computers in Victoria and the Australian Capital Territory was 
significantly greater than in Queensland. Among Year 10 students experience 
with computers was greater in South Australia than in Western Australia, New 
South Wales, the Northern Territory, Queensland and the Australian Capital 
Territory. Victorian and Tasmanian Year 10 students had greater experience of 
computers than their counterparts in Queensland. Across both Year 6 and Year 
10 it appears that experience of using computers is relatively high in South 
Australia and low in Queensland. 
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Table 5.3: Percentage of Students with more than Five Years Experience of Using Computers by 
State and Territory 





Year 6 


Year 10 


% with >5 
years usage 


Confidence 

Interval 


% with >5 
years usage 


Confidence 

Interval 


New South Wales 


53.0 


(±4.9) 


61.3 


(±4.9) 


Victoria 


60.4 


(±7.0) 


69.9 


(±4.3) 


Queensland 


47.8 


(±4.5) 


58.5 


(±4.8) 


South Australia 


63.6 


(±3.8) 


74.5 


(±2.9) 


Western Australia 


50.7 


(±5.6) 


63.4 


(±4.7) 


Tasmania 


51.9 


(±4.5) 


68.3 


(±4.4) 


Northern Territory 


49.6 


(±8.3) 


59.2 


(±9.1) 


Australian Capital Territory 


59.1 


(±5.7) 


58.4 


(±6.2) 



There are some significant differences associated with socioeconomic 
background. Relevant data are recorded in Table 5.4. The differences among the 
four groups are not statistically significant but if the four groups are collapsed 
to two the differences between the groups that could be labelled as “unskilled 
or skilled office and trade” and “professional or managerial” are statistically 
significant at both Year 6 and Year 10. It can be concluded from these data that 
socioeconomic differences in computer experience are between “unskilled or 
skilled office and trade” and “professional or managerial” and that difference 
exists at both Year 6 and Year 10. 



Table 5.4: Percentage of Students with more than Five Years Experience of Using Computers by 
Socioeconomic Group 





Year 6 


Year 10 


% with >5 
years usage 


Confidence 

Interval 


% with >5 
years usage 


Confidence 

Interval 


Four occupational groups 




Unskilled manual, office and 
sales 


52.0 


(±4.7) 


59.0 


(±6.3) 


Skilled trades, clerical & sales 


49.9 


(±4.9) 


62.1 


(±3.6) 


Other managers & associate 
professionals 


59.3 


(±4.3) 


67.7 


(±3.3) 


Senior managers & professionals 


58.9 


(±6.0) 


67.1 


(±5.0) 1 


Two occupational groups 




Skilled or unskilled trades and 
office 


50.9 


(±3.6) 


60.9 


(±3.2) 


Professional and managerial 


59.2 


(±3.6) 


67.5 


ST 
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There are some other differences in computer experience that are only 
statistically significant at Year 10. At Year 10, but not at Year 6, there is a higher 
percentage of non-Indigenous students than Indigenous students that have 
more than five years of computer experience (65 per cent (± 2.3) compared 
to 41.7 per cent (± 9.9). Also at Year 10 only there is a higher percentage of 
computer use by students for whom English is the main home language (66 per 
cent (± 2.7) compared to students for whom their home language is other than 
English (58 per cent (± 4.0) 



Variations in frequency of using computers 

In neither Year 6 nor Year 10 is there any significant difference in the frequency 
of using computers at home or at school between students for whom their 
home language is English and those for whom their home language is other 
than English. 

There is a set of characteristics associated with differences in the use of 
computers at home, but not at school, among Year 10 students but not among 
Year 6 students: sex. Indigenous status and geographic location. 

There is no significant difference in the frequency of computer use at home or 
at school between males and females among Year 6 students. However, there is 
a significant difference between male and female computer use at home among 
Year 10 students. There is no difference in school usage. Relevant data are 
recorded in Table 5.5. 

There is a similar pattern for Indigenous students with no significant difference 
between Indigenous and nondndigenous students at Year 6 in terms of either 
home or school use but with Indigenous students recording significantly lower 
home use than nondndigenous students at Year 10. Relevant data are recorded 
in Table 5.5. 



Table 5.5: Monthly Computer Usage for Females and Males, Indigenous and non-Indigenous 
Students and Different Locations among Year 10 Students 





Home use 


School Use 




Mean days 
per month 


Confidence 

Interval 


Mean days 
per month 


Confidence 

Interval 


Sex 




Females 


14.2 


(±0.46) 


9.1 


(±0.59) 


Males 


15.4 


(±0.46) 


97 _ 1 


(±0.44) J 


Indigenous Status 




Indigenous 


12.6 


(±1.54) 


11.0 1 


(±1.24)' "1 


Non-Indigenous 


14.9 


(±0.36) 


9.4 ) 


(±0.40) 1 


Geographic location 




Metropolitan 


15.3 


(±0.42) 


r“97i-n 


(±0.51 1 


Provincial 


13.7 


(±0.33) 


10.0 


(±0.59) 


Remote 


13.5 


(±0.65) 


10.8 1 


(±1.93) 
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In terms of geographic location the difference in home computer use is between 
metropolitan and either provincial or remote locations (there is no difference 
between provincial and remote locations). Relevant data are recorded in 
Table 5.5. Those data suggest there is a divide between metropolitan and non- 
metropolitan locations in terms of home use of computers. However, other 
data in Table 5.5 show that there is no significant difference in the frequency 
of computer use at school between Year 10 students from different geographic 
locations. 

There are differences in the frequency of computer use among socioeconomic 
groups. Relevant data are recorded in Table 5.6. 

Year 10 students whose parents were “senior managers and professionals” 
recorded significantly higher levels of computer use at home than those 
students whose parents were in either “unskilled manual, office or sales” or 
“skilled trades, clerical and sales” occupational groups. Among Year 6 students 
the statistically significant difference was between students whose parents 
were “senior managers and professionals” and those students whose parents 
were in “unskilled manual, office or sales” or occupations. There were no 
differences in school use among these socioeconomic groups at either Year 6 
or Year 10. 

It seems reasonable to conclude that differences in computer use between the 
highest and lowest socioeconomic groups are evident but are not as large as 
might have been expected. Fifty one per cent (± 7%) of Year 10 students whose 
parents were employed in “unskilled manual, office or sales” occupations used 
a computer at home every day compared to 67 per cent (± 5 %) of Year 10 
students whose parents were employed as “senior managers and professionals”. 
The corresponding comparison among Year 6 students was 39 per cent (± 5%) 
among students whose parents were from the “unskilled manual, office or sales” 
group and 46 per cent (± 5%) from the “senior managers and professionals” 
group. 
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Table 5.6: Index of Frequency of Use of Computers at Home and School by Socioeconomic 
Group 





Home use 


School use 


Mean days 
per month 


Confidence 

Interval 


Mean days 
per month 


Confidence 

Interval 


Year 6 students 




Unskilled manual, office and 
sales 


11.9 


(±0.7) 


9.5 


(±0.5) 


Skilled trades, clerical & sales 


12.9 


(±0.7) 


9.4 


II (±0.5) 


Other managers & associate 
professionals 


13.4 


(±0.6) 


9.7 


(±0.5) 


Senior managers & professionals 


13.6 


(±0.7) 


9.7 


II (±0.7) 


Year 10 students 




Unskilled manual, office and 
sales 


13.7 


(±1.24) 


9.2 


(±0.67) 


Skilled trades, clerical & sales 


14.4 


(±0.61) 


9.1 


i (±0.55) 


Other managers & associate 
professionals 


15.2 


(±0.48) 


9.5 


(±0.55) 


Senior managers & professionals 


15.9 


(±0.63) 


10.1 


il (±0.73) 



There are differences among States and Territories in the frequency with which 
students use computers in school. Table 5.7 shows a comparison of means for 
the reported frequency of using computers by Year 10 students. 



Table 5.7: Frequency of Use of Computers at School among Year 10 Students by State and 
Territory 





Mean 

days 

per 

month 


Conf. 

Interval 


TAS 


SA 


VIC 


NT 


WA 


QLD 


ACT 


Tasmania 


12.7 


±0.6 
















South Australia 


12.0 


±0.8 


• 














Victoria 


11.6 


±1.1 


• 


• 












Northern Territory 


11.0 


±2.4 


• 


• 


□1 










Western Australia 


9.6 


±1.0 


▼ 


▼ 


• 


• 








Queensland 


9.6 


±0.8 


▼ 


▼ 


▼ 








■ 


Australian Capital 
Territory 


9.2 


±0.8 


▼ 


▼ 


▼ 


▼ 


• 


• 


New South Wales 


7.0 


±0.6 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 


▼ 



Note: Read across the row to compare one jurisdictions mean with other jurisdictions. The 
source and comparison jurisdictions are listed as the row and column headings respectively. 

Legend: • denotes no significant difference in mean usage levels 

▼ denotes mean is significantly lower than the comparison jurisdiction 
▲ denotes mean is significantly higher than the comparison jurisdiction 



Those data indicate a substantial variation in usage of computers at school 
by Year 10 students from the highest using jurisdictions (Tasmania, South 
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Australia and Victoria) to the lowest (New South Wales)^“. The data shown 
in Table 5.8 for both Year 6 and Year 10 and for school and home use show 
much less variation among jurisdictions at Year 6. For school use the only 
statistically significant difference at Year 6 is between the highest using State 
(South Australia) and the lowest using State (New South Wales). The other 
differences are not statistically significant even though the pattern for Year 6 
is similar to that for Year 10 (the correlation coefficient for the two sets of use 
data is 0.87). 



Table 5.8: Index of Frequency of Use of Computers at Home and School by State and Territory 





Home use 


School use 


Mean days 
per month 


Confidence 

Interval 


Mean days 
per month 


Confidence 

Interval 


Year 6 students 




New South Wales 


12.7 


(±0.8) 


8.8 


(±0.8) 


Victoria 


13.4 


(±0.6) 


10.4 


(±0.9) 


Queensland 


12.7 


(±0.7) 


9.6 


(±0.6) 


South Australia 


12.3 


(±0.5) 


10.5 


(±0.7) 


Western Australia 


12.4 


(±0.5) 


9.6 


(±0.6) 


Tasmania 


12.8 


(±0.7) 


10.3 


(±0.9) 


Northern Territory 


10.8 


(±0.8) 


10.3 


(±1.0) 


Australian Capital Territory 


14.2 


(±0.6) 


10.2 


(±2.0) 


Year 10 students 




New South Wales 


14.9 


(±0.80) 


7.0 


(±0.61) 


Victoria 


15.6 


(±0.59) 


11.6 


(±1.06) 


Queensland 


14.8 


(±0.69) 


9.6 


(±0.79) 


South Australia 


14.7 


(±0.63) 


12.0 


(±0.77) 


Western Australia 


13.2 


(±0.95) 


9.6 


(±1.00) 


Tasmania 


13.2 


(±0.93) 


12.7 


(±0.63) 


Northern Territory 


14.1 


(±1.74) 


11.0 


(±2.35) 


Australian Capital Territory 


16.2 


(±1.14) 


9.2 


(±0.78) 



Table 5.8 also records the mean days per month on which students report 
using computers at home. At Year 6 there are some significant differences in 
home computer use. Home computer use in the Australian Capital Territory 
is significantly greater than in every jurisdiction other than Victoria. Home 
computer use by Year 6 students is significantly lower in the Northern Territory 
than in every other jurisdiction, in addition to these differences the data in 



20 Data from the OECD Programme for International Student Assessment indicate differences 
among jurisdictions in the ratio of computers to students in secondary school (OECD, 2004; 
2006). For Australia as a whole the ratio is 0.28. Jurisdictional ratios range from 0.23 in New 
South Wales, through 0.26 in the Australian Capital Territory, 0.28 in Queensland, 0.29 in 
Western Australia, 0.31 in Tasmania and South Australia, to 0.33 in Victoria and 0.35 in the 
Northern Territory. If the Australian Capital Territory is excluded there is a strong correlation 
between these data and the usage data in the present study (r=0.70). In the ACT reported 
school usage is lower than would be expected on the basis of availability. 
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Table 5.8 show that home computer use in Victoria is significantly greater than 
in South Australia. 

Among Year 10 students there are fewer differences in the level of home 
computer use. At Year 10 home computer use in the Australian Capital Territory 
and Victoria is significantly greater than in Western Australia and Tasmania but 
there are no other statistically significant differences among jurisdictions. There 
is a modest association between levels of home computer use for jurisdictions 
at Year 6 and Year 10 (the correlation coefficient is 0.59) and between levels of 
school computer use at Year 6 and Year 10 (the correlation coefficient is 0.88). 
However, there is little association between jurisdictional levels of computer 
use at home and computer use at school for either Year 6 or Year 10. 



Computer Platforms 

As shown by the data in Table 5.9, the overwhelming majority of students 
reported using either windows-based computers or both windows-based and 
Macintosh computers. Taken together this amounted to 83 per cent of Year 6 
students and 93 per cent of Year 10 students for their home computer use. if 
this is expressed as a percentage of those students who have a computer at 
home the percentages become 87 per cent and 95 per cent. The corresponding 
figures for school computer use were 83 per cent and 92 per cent respectively 
(or 84 and 93 per cent of this with access to a computer). The percentages 
using only Macintosh computers at home were 2.4 per cent in Year 6 and 1.3 
per cent in Year 10. At school the corresponding figures were 6.8 per cent 
and 4.2 per cent. Very few students used only a Macintosh at both home and 
school. 



Table 5.9: Computer platforms used by Year 6 and Year 10 Students 



Year 6 



79.4 


2.4 


4.1 


10.1 


4.1 


78.5 


6.8 


4.7 


9.3 


0.6 


(±1.9) 


(±0.7) 


(±1.0) 


(±1.2) 


(±1.3) 


(±3.8) 


(±2.7) 


(±1.5) 


(±1.5) 


(±0.3) 


Year 10 


89.8 


1.3 


3.1 


3.1 


2.6 


80.4 


4.2 


11.7 


2.6 


1.1 


(±1.3) 


(±0.4) 


(±0.8) 


(±0.7) 


(±0.9) 


(±4.5) 


(±2.1) 


(±3.2) 


(±0.8) 


(±0.5) 



Confidence intervals (95%) are shown in parentheses 



Percentages using each platform at school 



Windows Macintosh Both Other None 



Percentages using each platform at home 



Windows Macintosh Both Other None 
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Computer Applications 

As part of the computer-based student survey students indicated the extent to 
which they used various computer applications. To simplify the representation 
of these data the index based on days per 20-day working month of application 
use has been recorded. Table 5.10 records values of the index of use for males 
and females in Year 6 and Year 10. 



Table 5.10: Use of Computer Applications by Students in Year 6 and Year 10 





Male 


Female 


1 Persons 


Mean 


Conf. 

Interval 


Mean 


Conf. 

Interval 


Mean 


Conf. 

Interval 


Year 6 Students 




Play games on a 
computer 


12.51 


(±0.56) 


11.77 


(±0.46) 


12.19 


(±0.38) 


Use the internet to look 
up information 


9.42 


(±0.52) 


9.12 


(±0.53) 


9.29 


(±0.39) 


Use a computer for e- 
mail or “chatting” 


7.74 


(±0.57) 


9.33 


(±0.70)* 


8.54 


(±0.52) 


Use a computer to listen 
to music or watch DVDs 


8.15 


(±0.54) 


7.62 


(±0.49) 


7.85 


(±0.40) 


Use drawing, painting or 
graphics programs 


6.52 


(±0.46) 


8.49 


(±0.40)* 


7.47 


(±0.35) 


Do word processing 


6.58 


(±0.42) 


8.13 


(±0.50) 


7.34 


(±0.37) 


Download games or 
music from the Internet 


6.80 


(±0.46) 


5.03 


(±0.46)* 


5.89 


(±0.36) 


Use a computer for 
programming 


5.23 


(±0.44) 


4.70 


(±0.46) 


4.96 


(±0.35) 


Use mathematics, 
language or other 
learning programs 


4.55 


(±0.43) 


5.04 


(±0.56) 


4.76 


(±0.37) 


Use spreadsheets 


2.85 


(±0.27) 


2.82 


(±0.29) 


2.81 


(±0.20) 


Year 10 Students 




Use a computer for e- 
mail or “chatting” 


12.81 


(±0.52) 


13.30 


(±0.47) 


13.07 


(±0.37) 


Use a computer to listen 
to music or watch DVDs 


12.76 


(±0.54) 


11.33 


(±0.55)* 


12.09 


(±0.47) 


Use the internet to look 
up information 


12.43 


(±0.51) 


11.23 


(±0.47)* 


11.86 


(±0.36) 


Download games or 
music from the Internet 


10.41 


(±0.53) 


7.54 


(±0.49)* 


9.05 


(±0.37) 


Do word processing 


8.18 


(±0.45) 


9.38 


(±0.44) 


8.75 


(±0.35) 


Play games on a 
computer 


10.50 


(±0.52) 


5.71 


(±0.47)* 


8.25 


(±0.40) 


Use drawing, painting or 
graphics programs 


6.53 


(±0.50) 


5.75 


(±0.42) 


6.15 


(±0.31) 


Use a computer for 
programming 


3.64 


(±0.43) 


2.20 


(±0.26)* 


2.96 


(±0.26) 


Use spreadsheets 


2.47 


(±0.22) 


1.90 


(±0.17)* 


2.20 


(±0.14) 


Use mathematics, 
language or other 
learning programs 


2.06 


(±0.25) 


1.80 


(±0.26) 


1.94 


(±0.19) 



Note: * indicates that the difference between males and females is statistically significant 
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From Table 5. 1 0 it is possible to identify the most used and least used computer 
applications. At both Year 6 and Year 10 the three least used applications are 
computer programming, spreadsheet applications and education programs. 

At Year 6 by far the most frequently used application is “playing games on a 
computer” followed by using “the internet to look up information” and using “a 
computer for e-mail or chatting”. By far the least frequently used application 
is “using spreadsheets” followed by using “mathematics, language or other 
learning programs” and then using “a computer for programming” (e.g. Logo 
or HTML). 

Among Year 10 students the most frequently used applications are (and they 
are used with similar frequency): using a computer for e-mail or “chatting”, 
“downloading games or music from the internet” and “use the internet to look 
up information”. The least frequently used applications are using “mathematics, 
language or other learning programs” followed by “using spreadsheets” and 
using “a computer for programming”. 

From Table 5.10 it is possible to identify those applications that increase in use 
between Year 6 and Year 10 and those that decline in use. There is increased 
frequency of use, in Year 10 compared to Year 6, of using “a computer for e- 
mail or “chatting”, using “a computer to listen to music or watch DVDs” and 
using “the internet to look up information”, “downloading games or music 
from the internet”, and “doing word processing” (but this last by only a small 
amount). There is decreased frequency of use, in Year 10 compared to Year 
6, of “playing games on a computer”, using “mathematics, language or other 
learning programs”, using “a computer for programming”, and using “drawing, 
painting or graphics programs”. There is no difference between Year 10 and 
Year 6 in the low level of “using spreadsheets”. 

There are some differences in the use of computer applications by males and 
females, in Year 6, females “use drawing, painting or graphics programs”, “do 
word processing” and “use a computer for e-mail or “chatting” more frequently 
than males. On the other hand males “download games or music from the 
Internet” more frequently than females, in Year 10 males “play games on a 
computer” (and this difference is large), “download games or music from the 
Internet”, “use a computer for programming”, “use a computer to listen to music 
or watch DVDs”, and “use the internet to look up information” more frequently 
than females. Females in Year 10 use a computer to “do word processing” more 
frequently than males in Year 10. in Year 10 there is no significant difference 
between males and females in the frequency with which they “use a computer 
for e-mail or “chatting”, “use drawing, painting or graphics programs”, “use 
spreadsheets”, or “use mathematics, language or other learning programs”. 
The last three of these applications are used with low frequency by both males 
and females. 
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Attitudes to Computers 

In the student computer-based survey that accompanied the assessment 
instrument students responded to four questions about their attitudes to 
aspects of computing that were taken from the PISA survey of 2003. The 
percentages of males and females at Year 6 and Year 10 who strongly disagreed 
or disagreed, agreed and strongly agreed with the statements are recorded in 
Table 5.11. 

In interpreting these data attention is focused on the percentage of students 
who “strongly agreed” with each of the four statements. The first observation 
is that males and females from both Year 6 and Year 10 are positive about using 
computers. The percentages who strongly agreed with the statements ranged 
from 23 per cent of Year 10 students who strongly agreed with the statement 
that “I use a computer because I am very interested” to 58 per cent of Year 
6 students who strongly agreed with the statement that “I think playing or 
working with a computer is fun”. 

The second observation is that males expressed more favourable attitudes to 
computers than females. This applied to all items except for Year 10 responses 
to the statement “I lose track of time when I am working with a computer”. The 
strongest differences at Year 6 were on the items “I use a computer because 
I am very interested” and “I lose track of time when I am working with a 
computer”. At Year 10 the largest difference between males and females were 
on the items “I think playing or working with a computer is fun” and “I use a 
computer because I am very interested”. 
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Table 5.11: Attitudes of Year 6 and Year 10 Students towards Computers 





Percentages of Students in Each Response Category 
(confidence intervals below in parentheses) 




Males 


Females 




disagree 


agree 


Strongly 

agree 


disagree 


agree 


strongly 

agree 


Year 6 


It is very important to me 


15.0 


54.0 


31.1 


15.6 


61.8 


22.5 


to work with a computer 


(±2.2) 


(±3.1) 


(±2.8) 


(±2.2) 


(±3.0) 


(±2.9) 


I think playing or working 


4.1 


33.8 


62.1 


3.9 


41.4 


54.7 


with a computer is fun 


(±1.4) 


(±3.0) 


(±3.4) 


(±1.2) 


(±3.2) 


(±3.4) 


I use a computer because 


17.7 


42.5 


39.8 


21.6 


49.3 


29.1 


I am very interested 


(±3.0) 


(±3.3) 


(±3.1) 


(±2.9) 


(±3.0) 


(±2.8) 


I lose track of time when 
I am working with a 
computer 


17.7 


42.5 


39.8 


21.6 


49.3 


29.1 


(±3.0) 


(±3.3) 


(±3.1) 


(±2.9) 


(±3.0) 


(±2.8) 


Year 10 


It is very important to me 


12.2 


46.5 


41.3 


15.9 


55.6 


28.5 


to work with a computer 


(±2.2) 


(±3.3) 


(±3.7) 


(±2.2) 


(±3.3) 


(±3.4) 


1 think playing or working 


9.3 


49.5 


41.1 


15.0 


58.8 


26.2 


with a computer is fun 


(±1.9) 


(±3.0) 


(±3.8) 


(±2.5) 


(±3.1) 


(±2.6) 


I use a computer because 


23.3 


45.7 


30.9 


36.7 


47.3 


15.9 


1 am very interested 


(±2.8) 


(±2.6) 


(±2.8) 


(±3.3) 


(±2.9) 


(±2.5) 


1 lose track of time when 
I am working with a 
computer 


33.9 


37.3 


28.8 


32.4 


37.4 


30.2 


(±2.7) 


(±2.8) 


(±2.9) 


(±2.9) 


(±3.0) 


(±2.7) 



Confidence intervals (95%) are shown in parentheses 



Familiarity with Computers and ICT 
literacy 

In Chapter 4 the influence of student characteristics and jurisdiction on 
student ICT literacy is examined, using multiple regression analysis. That 
analysis provides an indication of the net effect of each variable or block of 
variables on ICT literacy scores, after allowing for the effects of associated 
variables. The analysis generates coefficients (B) that provide an indication of 
the net influences of the predictor or independent variables in the analysis on 
the dependent variable (student performance). The larger the (B) coefficient 
is, the stronger the effect of that variable is as a predictor on the dependent 
variable. The analysis also indicates the percentage of the variance explained 
by the blocks of variables on ICT literacy. Greater detail about the procedure 
is provided in Chapter 4. 
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In this chapter that analysis is extended by adding variables reflecting student 
familiarity with ICT. These variables reflect the years of using a computer^\ 
days each month of home computer use^^ days each month of school computer 
use and attitudes to computers (based on responses to three items concerned 
with working with computer, feeling that playing or working with a computer 
is fun, and being interested in using computers^^. 

In the analysis student characteristics were entered as block one, items 
concerned with familiarity with computers are entered as block two and 
jurisdictional variables are entered as block three. This is done so that the 
influence of computer familiarity can be examined after allowing for the effects 
of student characteristics and so that the effects of jurisdictional differences 
can be examined net of differences in student background and familiarity with 
computers. The analyses are reported separately for Year 6 and Year 10 in 
Table 5.12. The coefficients shown are those for the “full” or “complete” model. 
The percentage of variance explained by each block of variables is recorded 
for the sequence in which the blocks were included in the analysis. 

For both Year 6 and Year 10 the set of variables included in these analyses ac- 
counts for 22 per cent of the variance in student ICT literacy scores. It can be 
observed that student background accounts for 12 per cent of the variance in 
ICT literacy scores in Year 6 and 7 per cent of the variance in ICT literacy scores 
in Year 10. Familiarity with computers accounts for 8 per cent of the variance 
in ICT literacy scores in Year 6 and 13 per cent of the variance in ICT literacy 
scores in Year 10. In other words familiarity is a more important influence for 
Year 10 students than Year 6 students possibly reflecting a greater range of ex- 
periences with computers among students in the higher Year level. In contrast, 
student background is a more important influence on ICT literacy for Year 6 
students than Year 10 students. For both Year 6 and Year 10 only between 1 and 
2 per cent of the variance in ICT literacy is associated with the jurisdiction in 
which the student lived and attended school. This reflects the greater variation 
between students within each jurisdiction than between jurisdictions. 

Among the student background characteristics the strongest influence on ICT 
literacy is socioeconomic background. The net difference between the average 
ICT literacy scores of Year 6 students whose parental occupational group is 
classified as “unskilled manual, office and sales” and students whose parental 
occupational group is “senior manager or professional” was 73 scale points. In 
Year 10 the corresponding difference is a little less, being just 51 scale points. 



21 On a five point scale from “never”, through “less than one year”, “one to three years”, “three to 
five years”, to “more than five years”. 

22 Estimated from responses provided by students on a scale from “never”, through “less than 
once a month”, “between once a week and once a month”, “a few times each week” to “almost 
every day”. 

23 The items are “It is very important to me to work with a computer”, “To play or work with a 
computer is really fun”, and “1 use a computer because 1 am interested”. Students responded 
on a four point scale from “strongly disagree” through “disagree” and “agree’ to “strongly 
agree”. The responses were weighted using coefficients derived from a confirmatory factor 
analysis. 
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The net difference between Indigenous and non-Indigenous students is 
approximately 36 scale points at both Year 6 and Year 10. As noted previously 
this is a smaller difference than is typically observed in other assessment 
domains. There is also a net difference between students in remote geographic 
locations and those in metropolitan locations with the gap being between 22 
(Year 6) and 33 (Year 10) scale points. Females score higher than males by 
approximately 10 scale points at both Year levels. 



Table 5.12: Results of Regression Analysis of ICT Literacy on Student Characteristics 





Year 6 Students 


Year 10 Students 








Variance 






Variance 


Predictor 


Regression 

Coefficient 

(B) 


Conf. 

Interval 


Explained 

(R 

squared) 


Regression 

Coefficient 


Conf. 

Interval 


Explained 

(R 

squared) 


Intercept 

bO 


273.8 


±128.6 




905.9 


±138.7 




Block 1 Student background 












Age 


-0.8 


±10.5 




-31.4 


±9.1 




Sex 


9.5 


±7.7 




10.7 


±7.6 




Indigenous 


-35.3 


±20.8 




-36.4 


±21.8 




Language other 
than English 


-8.2 


±10.5 




-7.0 


±9.7 




Senior 














managers & 
professionals 
Managers 
& associate 


73.2 

47.9 


±11.9 

±10.2 


12.5% 


51.1 

30.5 


±12.6 

±10.7 


7.4% 


professionals 














Tradespeople, 
skilled office, 
sales service 


25.5 


±9.4 




17.7 


±10.5 




Provincial 

location 


-14.7 


±12.0 




-2.4 


±11.9 




Remote location 


-22.3 


±19.9 




-33.1 


±24.5 




Block 2 Computer familiarity 












Computer 

experience 


8.0 


±3.0 




13.1 


±3.2 




Home usage 
School usage 


1.5 

1.1 


±0.5 

±1.0 


7.6% 


2.2 

0.7 


±0.5 

±0.7 


13.0% 


Attitudes to 
computers 


13.1 


±4.6 




6.4 


±4.4 




I Block 3 Jurisdiction (compared to Queensland students) 






New South 
Wales 


28.5 


±14.5 




22.4 


±14.9 




Victoria 


39.6 


±14.9 




26.3 


±13.8 




South Australia 


27.3 


±14.8 




9.5 


±14.2 




Western 

Australia 

Tasmania 


4.7 

35.6 


±10.9 

±19.5 


1.6% 


-9.6 

15.3 


±13.0 

±15.9 


1.3% 


Northern 

Territory 

Australian 


13.6 


±20.3 




-3.2 


±28.9 




Capital 

Territory 


34.4 


±21.8 




34.5 


±19.6 




Full model 






21.7% 






21.7% 



Notes: 

Regression coefficients in bold are significant (a< .05). 

When categorical variables involving more than two categories are included in a regression 
analysis it is necessary to designate one category as the reference category and exclude that 
from the analysis so that results are relative to the reference category. In these analyses the 
reference category for socioeconomic group was unskilled labourers, office, sales and service 
staff, for location the reference category was “metropolitan” and for jurisdiction the reference 
category was Queensland. 
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It is interesting that the various aspects of computer familiarity had such 
an influence on ICT literacy. On the basis of the data in Table 5.12 the net 
difference (meaning other aspects being the same) between a person having 
used a computer for “one to three years” and having used a computer for 
“three to five years”(i.e. one point on the scale) is between eight (Year 6) and 
13 scale points (Year 10). For every five additional days on which a computer 
is used per month there is a net effect on ICT literacy of between seven (Year 6) 
and 11 (Year 10) scale points. Days of school use have less of an impact being 
less than six points in Year 6 and less than four points in Year 10 (which is not 
statistically significant). Attitudes to computers have a stronger impact on ICT 
literacy at Year 6 than at Year 10 but the magnitude is harder to interpret in 
concrete terms. 

Table 5.12 also displays the net effects for jurisdictions. Overall, it can be seen 
that the net differences among jurisdictions in this table are smaller than in 
the corresponding Table 4.18. This is because some of the differences among 
jurisdictions are a consequence of differences in computer familiarity. The 
analyses reported in Table 5.12 make allowance for differences in computer 
familiarity. This does not mean that the originally reported differences are not 
real but simply that they can be interpreted as partly reflecting differences in 
familiarity with computers. 

In Year 6 the jurisdiction that is most different from the reference jurisdiction 
(Queensland) is Victoria, followed by Tasmania, the Australian Capital Territory, 
New South Wales and South Australia. However the net effect for Victoria in 
Table 5.12 is 40 points compared to 50 points in the results in Table 4.18 where 
no allowance is made for computer familiarity. Victorian students have greater 
access to computers at home and school than their peers in Queensland and 
hence the adjustment has narrowed the gap. 

In Year 10 the jurisdiction that is most different from Queensland is the 
Australian Capital Territory, followed by Victoria, and then New South Wales. 
The net effect for the Australian Capital Territory is 34 points compared to the 
effect of 38 points when no adjustment for computer familiarity is made. 



Concluding Comments 

There are differences in the extent to which students in Years 6 and 10 have the 
opportunity to become familiar with computers. These differences are most 
evident in the differences in home computer usage between socioeconomic 
groups and in school computer usage among Year 10 students. Furthermore 
these differences appear to impact on ICT literacy scores and contribute to 
part, but not all of, the variations in ICT literacy among students. There is an 
argument that can be sustained by the results of the analyses of data in this 
chapter that reducing the variations in ICT literacy among school students will 
require some attention to differences in familiarity and therefore in access to 
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computers. It is also evident that students vary considerably in the computer 
applications that they use. Those patterns of use differ between Year 6 and 
Year 10, and between males and females. Communication is a frequent use 
at both Year 6 and Year 10 and using the internet to look up information is 
also a frequent application at both Year levels. However, there was much less 
frequent use of applications that involved creating, analysing or transforming 
information. The lack of use of these types of application appears to be reflected 
in the aspects of ICT literacy that are less evident in the responses of students 
to the tasks that they were asked to complete. 



91 



92 



Chapter 6 
Conclusion 



Over a short period of time ICT has become a pervasive part of society that 
has changed the ways in which people communicate, altered the ways in 
which data are accessed and processed, and redefined the bases of many 
occupations. Proficiency in ICT has become important for life in modern society 
and developing ICT literacy has become an important goal of many school 
systems. In Australia the national goals for schooling include the goal that when 
students leave school they should be: confident, creative and productive users 
of new technologies, particularly information and communication technologies, 
and understand the impact of those technologies on society (MCEETYA, 1999: 
Goal 1.6). 

Sometimes the view is expressed that young people use ICT frequently and 
with such facility that their development of ICT literacy takes place through 
the regular activities in which they engage. Overall, young people are frequent 
users of ICT and adults who have not grown up with contemporary ICT tend to 
assume that young people understand a great deal about how, when and why to 
apply those technologies. The results of this assessment survey indicate that 
ICT literacy is not developed to a uniformly high level among school students. 
For that reason they also suggest that monitoring ICT literacy should continue 
to be an important element of a National Assessment Program. Although there 
is evidence from this survey and other sources that indicates a high level of 
use of ICT by school students, it appears that there are aspects of using ICT 
for communicating, creating and sharing information that are learned through 
systematic teaching rather than incidental use. 
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Defining ICT Literacy 

ICT literacy is defined in a variety of ways from those definitions that emphasise 
computer skills and knowledge of computer systems through those definitions 
that stress information processing capacities to those definitions that focus 
broadly on associated research and inquiry methods. The definition adopted 
by MCEETYA as the basis for its National Assessment Program emphasises real- 
world application to relatively high order processes in a range of contexts, it 
defines iCT literacy as the ability of individuals to use ICT appropriately to access, 
manage, integrate and evaluate information, develop new understandings, and 
communicate with others in order to participate effectively in society (MCEETYA, 
2005). 

it is recognised that ICT literacy is a broader concept than computer literacy 
but in this assessment the focus is on the use of computers. One important 
reason for this focus is that there is a reasonable basis for viewing computer 
literacy as one construct or dimension. The results of the assessment support 
the proposition that there is one construct underpinning the computer 
based tasks that form the basis of the assessment and that this can be called 
computer literacy. There is no certainty that one construct would underpin a 
range of other information technologies. On the basis of informal observation 
one would not necessarily expect that young people who are adept at using 
mobile telephones for text messaging and other applications would necessarily 
be proficient users of computer technologies, in addition, learning about and 
with computers is a common experience to which one could reasonably expect 
students to have had deliberate and considered exposure in schools. Other 
forms of iCT are currently less closely connected to what happens in school 
and are less widely recognised as the business of schools. 

The accepted definition of iCT literacy in the MCEETYA National Assessment 
Program brings into focus the relationship between computer skills and 
computer-based communication. The data from the national assessment show 
that a high proportion of Year 6 and 10 students can complete concrete, skills- 
based computer tasks using conventional software. A smaller proportion of Year 
6 and 10 students are able to use software functions creatively to reconstruct 
information for particular communicative purposes, in particular, a relatively 
small proportion of students show evidence of planning in the structure and in 
the use of software features such as formatting of their information products, 
it is likely that simply providing students with the opportunity to complete 
assignment work using computers does not result in them developing 
understandings of how the iCT medium and communicative conventions can 
be applied to support the communicative intention of their work. 

The data show significant growth in iCT literacy between Year 6 and Year 10 
independent of other influences such as frequency of and attitudes towards 
computer use that were measured in the student survey. This suggests that 
student learning may be influenced by their exposure to school iCT programs 
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and those teaching programs that focus on planning and implementing 
specific communicative tasks using computers are likely to improve students’ 
facility in this area. As ICT increasingly utilises automated applications (such 
as automatically correcting spelling or saving files with multiple retrieval 
references), the importance of planning and consideration of communicative 
purpose relative to skills application is likely to assume increased 
importance. 



Factors Associated with ICT Literacy 

Student background characteristics are related to ICT literacy and the patterns 
of those relationships are similar in Year 6 and Year 10. Socioeconomic 
background is the characteristic that has the largest effect on ICT literacy. In 
Year 6, 32 per cent of students whose parents are from the “unskilled manual, 
office and sales” occupational groups attain the proficient standard compared 
to 68 per cent of students whose parents are from the “senior managers and 
professionals” occupational group. In Year 10 the corresponding figures are 49 
per cent and 75 per cent. These are manifestations of a substantial difference 
in an important outcome of school education that are partly, but not entirely, 
associated with differences students experience and frequency of using 
computers. There is a substantial gap in the ICT literacy of Indigenous and 
non-Indigenous students. In Year 6, 30 per cent of Indigenous students attained 
the proficient standard compared to 50 per cent of non-Indigenous students. 
At Year 10, the corresponding percentages were 35 per cent and 62 per cent. 
There was also evidence of disadvantage in the development of ICT literacy, in 
both Year 6 and Year 10, for students from remote locations. 

The data relating to ICT literacy achievement across socioeconomic. 
Indigenous and non-Indigenous and school location groups largely parallel 
those in literacy, numeracy, civics and citizenship and science and, as in those 
learning areas, consideration should be given about how best to reduce the 
achievement divide associated with these student background factors. 

There are small differences between males and females at Year 6 (more females 
than males attained the proficient standard by seven percentage points) but 
not at Year 10 and there are no differences at all between students for whom a 
language other than English was mainly spoken and other students. 

Students’ experience of using computers and the frequency with which they 
report using computers at home (at both Year 10 and Year 6) and school (but 
a smaller influence at Year 10 than Year 6) influenced their ICT literacy. Not 
surprisingly there is an effect of familiarity with computers that affects ICT 
literacy. In these data there is support for the proposition that greater access 
to computing resources results in higher levels of ICT literacy. 
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In addition there is an association between attitudes to computers and ICT 
literacy. Students who are favourably disposed to working with computers 
attain higher levels of ICT literacy. Of course, the direction of causation is far 
from clear. It could be that enjoying working with computers results in higher 
levels of ICT literacy or it could be that higher levels of ICT literacy make 
working with computers more enjoyable. 

At Year 6, when the comparisons among jurisdictions including confidence 
intervals are taken into account, there appear to be three groups of jurisdictions 
in terms of ICT proficiency. Significantly more Victorian students attained the 
proficient standard (58 per cent) than the national level of 49 per cent. There 
was no significant difference between national attainment of the proficient 
standard and the performance of students in Tasmania, New South Wales and 
South Australia (49 to 52 per cent). In Queensland and Western Australia the 
proportion of students attaining the proficiency level was significantly lower 
(38 to 40 per cent) than the Australian average. Although the percentage of 
students attaining the proficient standard in the Australian Capital Territory 
(58 per cent) appear to be similar to the percentage in Victoria the difference 
from the national attainment is not statistically significant and one cannot be 
certain of the difference. Similarly the percentage of students attaining the 
proficient standard in the Northern Territory (36 per cent) would appear to 
be similar to the percentages for Queensland and Western Australia but one 
cannot be certain that the figure is statistically different form the national 
attainment at Year 6. 

For Year 10 there was no significant difference between the percentage of 
students attaining the proficient standard in any jurisdiction and the Australian 
average. The range was from 67 per cent of students in Victoria attaining the 
Year 10 proficient standard to 49 per cent in the Northern Territory and 56 
per cent in Western Australia but these differences were not statistically 
significant. 



Summary 

The National Assessment Program in ICT Literacy for 2005 was computer- 
based and based on the completion of specific skills and the conduct of larger 
tasks using authentic applications of software. The assessment operationalises 
a definition of ICT literacy as accessing, managing, integrating and evaluating 
information. 

Qverall, the results indicate that there is variation among students in ICT literacy. 
Qne should not assume that students are uniformly becoming adept because 
they use ICT so widely in their daily lives. The results of the assessment survey 
suggest that students use ICT in a relatively limited way and this is reflected 
in the overall level of ICT literacy. Communication with peers and using the 
internet to look up information are frequent applications but there is much less 
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frequent use of applications that involve creating, analysing or transforming 
information. Lack of familiarity with these latter types of application appears 
to be reflected in students’ ICT literacy. 

Overall, 49 per cent of Year 6 students attained the proficient standard for 
that Year level by being able to: “generate simple general search questions 
and select the best information source to meet a specific purpose, retrieve 
information from given electronic sources to answer specific, concrete 
questions, assemble information in a provided simple linear order to create 
information products, use conventionally recognised software commands to 
edit and reformat information products”. Sixty-one per cent of Year 10 students 
reached or exceeded the proficient standard for Year 10 by indicating that 
they were able to: “generate well targeted searches for electronic information 
sources and select relevant information from within sources to meet a specific 
purpose, create information products with simple linear structures and use 
software commands to edit and reformat information products in ways that 
demonstrate some consideration of audience and communicative purpose”. 

There are substantial differences between Year 6 and Year 10 suggesting that 
considerable growth in ICT proficiency takes place over these four years. 
Within each Year level there are differences associated with socioeconomic 
background. Indigenous status and remote geographic locations (compared to 
metropolitan locations). 
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Appendix 1 

Survey Design and Sampling 
Procedures 



Sampling 

The target populations for the study were Year 6 and Year 10 students enrolled 
in schools across Australia. The sample design of the National Assessment 
Program - Information and Communications Technology Literacy 2005 was a two- 
stage stratified cluster sample design, similar to that used by international 
assessments such as the Trends in International Mathematics and Science 
Study (TIMSS) and the OECD Programme for International Students Assessment 
(PISA). The first stage consists of a sample of schools, explicitly stratified 
according to state and sector and implicitly stratified by location. Within strata 
schools were selected with a probability proportional to size. The second 
stage consists of a random sample of 15 students from the target year level in 
sampled schools. Samples were drawn separately for Year 6 and Year 10. 



The Sampling Frame 

The national school sampling frame is a comprehensive list of all schools in 
Australia, developed by the Australian Council of Educational Research (ACER) 
by coordinating information from multiple sources, including the Australian 
Bureau of Statistics and State and Territory education department databases. 
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School exclusions 



For the purpose of this study, only schools containing Year 6 or Year 1 0 students 
were used. In addition, some schools were excluded from the possibility of 
being sampled. Schools excluded from the target population included non- 
mainstream schools (such as schools for students with intellectual disabilities 
or hospital schools), schools with fewer than five students in the target year 
levels and very remote schools. These exclusions account for 1.8 per cent of 
the Year 6 population and 0.8 per cent of the Year 10 population. 



Sample Design 

For both the year 6 and year 10 samples, sample sizes were determined that 
would provide accurate estimates of achievement outcomes for all states and 
territories (with 95 per cent confidence limits of +/- 0.15 standard deviations 
to +/-0.2 standard deviations for estimated means). This required an effective 
sample size (i.e., the sample size of a simple random sample that would 
produce the same precision as the complex sample design) in the larger states 
of around 200 students. A smaller sample size was required in the smaller 
states and territories because of the finite population correction factor (i.e. 
as the proportion of the total population surveyed approaches 100 per cent 
the precision of the sample increases for a given sample size). Further detail 
of the sample design is provided in the Information and Communications 
Technologies Literacy Survey 2005 Technical Report. 

Table Al.lshows the population of schools and students (net of schools 
excluded from the target population) and the planned sample. 



Table A 1.1: Year 6 and 10 Target Population and Planned Samples by State and Territory 



Year 6 Year 10 





Population 


Planned Sample 


1 Population 


Planned Sample 


Schools 


Students 


Schools 


Students 


Schools 


Students 


Schools 


Students 


NSW 


2368 


87905 


41 


591 


779 


81715 


41 


608 


VIC 


1841 


68235 


41 


593 


545 


60720 


40 


600 


QLD 


1385 


54885 


41 


596 


443 


51739 


40 


600 


WA 


877 


27749 


41 


598 


316 


27862 


41 


606 


SA 


628 


19373 


42 


603 


220 


18966 


41 


606 


TAS 


235 


6703 


31 


445 


100 


6153 


30 


450 


ACT 


140 


4721 


16 


234 


35 


4709 


15 


225 


NT 


104 


2896 


16 


239 


33 


2047 


15 


225 


Australia 


7578 


272467 


269 


3899 


2471 


253911 


263 


3920 
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Structural Differences in State and 
Territory Education Systems 

The sample, while designed to be representative of the student population, 
incorporates some structural differences that must be kept in mind when 
interpreting the results of the National Assessment Program - Information and 
Communications Technology Literacy. One important feature of the sample is 
that it is year-based in order to be consistent with the reporting of literacy 
and numeracy performance in the National Report on Schooling in Australia. 
However, due to differences in school starting age, the length of time students 
have spent in formal schooling varies between the States and territories. 
Table A1.2 shows the effect that the structural difference in Australian state 
and territory education systems have on the ages of students in the target 
populations. 



Table A1.2: Average Age at Assessment and Average Time at School by State and Territory 





Year 6 


Year 10 


Average age at 
assessment 


Average time at 
school 


Average age at 
assessment 


Average time at 
school 


NSW 


12 yrs Omths 


5yrs 1 Imths 


10 5 TS Omths 


Oyrs 1 Imths 


VIC 


12yrs Imths 


Oyrs Omths 


lOyrs Imths 


lOyrs Omths 


QLD 


1 lyrs Omths 


5yrs 1 Omths 


ISyrs Smths 


Oyrs 1 Omths 


SA 


llyrs 1 Imths 


Oyrs Smths 


ISyrs 1 Omths 


lOyrs 7mths 


WA 


1 lyrs Smths 


5yrs 1 Omths 


ISyrs Smths 


Oyrs 1 Omths 


TAS 


12yrs 2mths 


Oyrs Omths 


ISyrs 2mths 


lOyrs Omths 


NT 


llyrs lOmths 


Oyrs Smths 


ISyrs Omths 


lOyrs 4mths 


ACT 


12 yrs Omths 


0 yrs Smths 


IS yrs Omths 


10 yrs Smths 



Table A1.2 shows that there is 9 month difference in average age at testing 
between students in Western Australia (the ‘youngest’ state) and students in 
Tasmania (the ‘oldest’ state). Students in Western Australia and Queensland 
had also experienced almost one year of formal schooling less than students 
in Victoria and Tasmania. 



First sampling stage 

The school sample was selected from all non-excluded schools in Australia 
which had students in Year 6 or Year 10. Stratification by state was explicit, 
resulting in separate samples being drawn for each state. Stratification by 
sector and school size was implicit, resulting in the schools within each state 
being ordered by size (according to the number of students of the target year 
level) within a grouping by sector. The selection of schools was carried out 
using a systematic probability-proportional to size (PPS) method. 
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Replacement schools 

As each school was selected, the next school in the sampling frame was 
designated as a replacement school for use should the sampled school not 
participate. The school previous to the sampled school was the second 
replacement. It was used if neither the sampled school nor the first replacement 
participated. In some cases (such as secondary schools in the Northern 
Territory) there were not enough schools available for the replacement 
samples to be drawn. Because of the sorting of each explicit stratum by sector 
and size, the replacement schools were generally similar (with respect to size, 
state and sector) as the school for which they were a replacement. 



Second sampling stage 

The second stage of sampling a random sampling technique was used to 
select students within sampled schools. In most cases, 15 students, with 
three nominated replacements, were sampled from each sampled school, in 
schools where 15 or less students were available at the target year level, all 
students were automatically selected. In schools where more than 15 students 
were enrolled, students were randomly sampled with equal probability of 
selection. 

In the case of small schools, two schools were combined to make a pseudo- 
school group prior to sampling. For example, two schools with 13 and 15 Year 
6 students respectively might be combined into a single pseudo school of 28 
students. This was to maximise the number of students selected per school 
(the sample design was based on 15 students per school) and to minimise 
any unintended bias introduced through the non selection of small schools. 
Pseudo-schools were treated like other schools and had equal probability of 
selection during sampling. 



Participation 

Student exclusions 

Within the sampled classrooms, individual students were eligible to be 

exempted from the assessment on the basis of: 

• Functional Disability: Student has a moderate to severe permanent physical 
disability such that he/she cannot perform in an assessment situation. 

• Intellectual Disability: Student has a mental or emotional disability and 
is cognitively delayed such that he/she cannot perform in the assessment 
situation. 

• Limited Assessment Language Proficiency: The student is unable to read 
or speak the language of the assessment and would be unable to overcome 
the language barrier in the assessment situation. Typically a student 
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who has received less than one year of instruction in the language of the 
assessment would be excluded. 

In this survey school principals tended to exclude students with disabilities 
from the student list provided. Consequently these students were not presented 
for the assessment. The number of student-level exclusions at Year 6 was 134 
and at Year 10 the number was 189. The total student population exclusion 
rate was 3.6 per cent at Year 6 and 4.8 per cent at Year 10. 



Participation rates 

The Year 6 Australian school participation rate was 99% including replacement 
schools. Excluding replacement schools, the school participation rate was 
95%. At Year 10, the Australian school participation rate was 97% including 
replacement schools. Excluding replacement schools, the school participation 
rate was 91%. Tables A1.3 and A1.4 detail Year 6 and 10 school refusals and 
participation information, including the final participation rate for the states 
and territories. 



Table A 1.3: Year 6 Numbers and Percentages of Participating Schools by State and Territory 





Sample 


Excluded 

Schools 


Eligible 

Schools 


Participating 
Schools 
- Sampled 
Schools 


Participating 
Schools - 
Replacement 
Schools 


Non- 

Participating 

Schools 

(Refusals) 


Total 

Number of 
Participating 
Schools 


School 

Participation 

Rate* 


NSW 


41 


0 


41 


37 


1 


3 


38 


93% 


VIC 


41 


0 


41 


40 


1 


0 


41 


100% 


QLD 


41 


0 


41 


39 


2 


0 


41 


100% 


SA 


41 


0 


41 


41 


0 


0 


41 


100% 


WA 


41 


0 


41 


41 


0 


0 


41 


100% 


TAS 


31 


0 


31 


31 


0 


0 


31 


100% 


NT 


16 


0 


16 


15 


1 


0 


16 


100% 


ACT 


16 


0 


16 


10 


6 


0 


16 


100% 


Aust 


268 


0 


268 


254 


11 


3 


265 


99% 



* Participating replacement schools are included. 

Table A1.4: Year 10 Numbers and Percentages of Participating Schools by State and Territory 





Sample 


Excluded 

Schools 


Eligible 

Schools 


Participating 
Schools 
- Sampled 
Schools 


Participating 
Schools - 
Replacement 
Schools 


Non- 

Participating 

Schools 

(Refusals) 


Total 

Number of 
Participating 
Schools 


School 

Participation 

Rate* 


NSW 


41 


0 


41 


35 


4 


2 


39 


95% 


VIC 


40 


0 


40 


39 


1 


1 


40 


100% 


QLD 


40 


0 


40 


35 


4 


1 


39 


98% 


SA 


41 


1 


40 


39 


1 


0 


40 


98% 


WA 


41 


0 


41 


38 


2 


1 


40 


98% 


TAS 


30 


0 


30 


30 


0 


0 


30 


100% 


NT 


15 


0 


15 


11 


0 


4 


11 


73% 


ACT 


15 


0 


15 


13 


2 


0 


15 


100% 


Aust 


263 


0 


263 


240 


14 


9 


254 


97% 



* Participating replacement schools are included. 
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Of the eligible sampled students, 96 per cent of Year 6 students and 93 per cent 
of Year 10 students completed the assessment. Tables A1.5 and A1.6 detail the 
Year 6 and 10 student, absentee and participation information, including the 
final student and participation rates for the states and territories. 



Table A1.5: Year 6 Numbers and Percentages of Participating Students by State and Territory 





Number of 
sampled 
students in 
participating 
schools 


Number of 
Absentees 
(including 
parental refusal) 


Number of 
Participating 
students 


Student 

Participation 

Rate* 


NSW 


591 


57 


534 


90% 


VIC 


593 


18 


575 


97% 


QLD 


596 


22 


574 


96% 


SA 


598 


7 


591 


99% 


WA 


603 


33 


570 


95% 


TAS 


445 


-2 


447 


100% 


NT 


234 


3 


231 


99% 


ACT 


239 


15 


224 


94% 


Australia 


3899 


153 


3746 


96% 



* Percentage of participating eligible (non-excluded) students in the final sample. 

Table A1.6: Year 10 Numbers and Percentages of Participating Students by State and Territory 





Number of 
sampled 
students in 
participating 
schools 


Number of 
Absentees 
(including 
parental 
refusal2) 


Number of 
Participating 
students 


Student 

Participation 

Ratel 


NSW 


608 


67 


541 


89% 


VIC 


600 


7 


593 


99% 


QLD 


600 


38 


562 


94% 


SA 


606 


25 


581 


96% 


WA 


606 


49 


557 


92% 


TAS 


450 


22 


428 


95% 


NT 


225 


63 


162 


72% 


ACT 


225 


22 


203 


90% 


Australia 


3920 


293 


3627 


93% 



* Percentage of participating eligible (non-excluded) students in the final sample. 
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Survey Implementation 

The administration of the assessment, from the first point of contacting schools 
after sampling through to the preparation of the data for analysis, contains a 
number of steps that have to be undertaken by the contractor or the school. 
These are listed in order in Table A1.7 and further described in this chapter. 

Contact with schools 

The field administration of the National Assessment Program - Information 
and Communications Technology literacy required several approaches to the 
sampled schools to request or provide information: 

• The initial approach to the principals of the sampled schools to inform 
them of their selection. This included a request to name a School Contact, 
who would coordinate the assessment in the school, and a list of all of the 
Year 6 or Year 10 students in the school in an electronic form that was used 
to generate the student level sample. 

• If the sampled school declined to take part (even with encouragement 
from an education authority Liaison Officer), the replacement school was 
contacted. 

• ACER drew a random sample of 15 students from the school cohort, and 
3 random replacement students to allow for any casual absenteeism. 
Principals advised ACER if the selection included a student for whom the 
parent refused to allow participation, or an exempted student. These were 
replaced by from the remaining school pool. 

• School Contacts were sent the School Contact’s Manual together with 
notification of the selected students for that school. They were requested to 
advise the school’s preferred dates for testing (electronically via email). 

• Copies of the Test Administrator’s Manual were sent to the School Contact, 
to inform him/her of the procedures that would be implemented on the 
test. 

• All the assessment materials were provided in electronic form on laptops 
brought to schools by the Test Administrator on the assessment date. 
The School Contact was responsible to ensure that an adequate learning 
area was provided for a mini-lab of laptops to be constructed and also 
to provide the Test Administrator with the details required in the Student 
Participation Form. 

• The final contact with schools was to send them the results for the 
participating students and to thank them for their participation. 
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Table A1.7: Procedures for Field Administration 



Contractor Activity 


School Activity 


Contact sampled schools. 






Complete the Facsimile Response Form and 
remit the Year 6/10 cohort list in electronic 
format. 


Appoint school contact 




Sample drawn of 15 sampled students and 3 
replacement students from the cohort list. 




Notify schools of the selected students 
and provide them with the School Contact’s 
Manual and advise the school of the test 
administration window and request preferred 
test dates 






Confirm the preferred test date 


Allocate Test Administrator to the school 




Test Administrator make contact with school 
to finalise test arrangements and co-ordinate 
logistics 




Copy of the Assessment Administrator’s Manual 
to schools. 






Make arrangements for the assessment: 

• Organise an assessment room 

• Notify students and parents 


Test Administrator delivers all test materials 
and resources to the school and administers 
the instruments 






Ensure availability of students and inform 
Test Administrator regarding absenteeism, 
refusals and exemptions. 


Test Administrator records participation 
status on the Student Participation Forms; 
complete the Assessment Administration Form. 




Test Administrator makes physical backup of 
all student responses and remits to ACER 




Marking 




Data Entry 




Data Cleaning 




Create and send School Reports to the 
schools. 





At each of the steps that required information to be sent from the schools, a 
definite timeframe was provided for the provision of this information. If the 
school did not respond in the designated timeframe, follow-up contact was 
made via fax, email and telephone. 

In order to ensure the participation of sampled schools. Liaison Officers were 
appointed for each jurisdiction. The Liaison Officers were expected to facilitate 
communication between ACER and the schools selected in the sample from 
their respective jurisdiction. The Liaison Officers helped to achieve a high 
take-up rate for the assessment, which ensured valid and reliable data. 
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Information management 

In order to track schools and students, databases were constructed. 
The sample database identified the sampled schools and their matching 
replacement schools and also identified the participation status of each school. 
The schools database contained a record for each participating school and 
contained contact information as well as details about the School Contact and 
participating students. The student database contained student identification 
and participation information. The achievement database contained the final 
achievement and student background survey data. 

In order to track information in these databases, a system of IDs was used. The 
School ID comprised information about state and sector, as well as whether the 
school was a sampled or a replacement school, and a school number (unique 
within each state). The Student ID included the School and a student number 
(unique within each school). 

The School Contact 

Participating schools were asked to appoint a School Contact to coordinate 
the assessment within the school. The School Contact’s responsibilities were 
to: 

• Liaise with ACER on any issues relating to the assessment; 

• Provide ACER with student names for the sampled cohort; 

• Schedule the assessment and arrange a space for the session(s); 

• Check the Student Participation Form from ACER for errors; 

• Notify teachers, students, and parents about the assessment according to 
the school’s policies; 

• Liaise with the ACER Test Administrator; 

• Assist the Test Administrator as necessary; 

• Provide information as required for the completion of the administration 
forms; and 

• Arrange for the attendance of replacement students (as sampled) if 
required. 

Each School Contact was provided with a manual (the School Contact’s Manual) 
that described in detail what was required as well as providing a checklist 
of tasks and blank versions of all of the required forms. Detailed instructions 
were also provided regarding the participation and exclusion of students with 
disabilities and students from non-English speaking backgrounds. 

The Test Administrator 

Each school was required to appoint an Assessment Administrator(s). In 
most cases this was the regular class teacher. This was done to minimise the 
disruption to the normal class environment. 
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ACER appointed a Test Administrator to deliver the National Assessment 
Program - Information and Communications Technology literacy in each school, 
according to the standardised administration procedures provided in the Test 
Administrator’s Manual. The Test Administrator had also to complete the 
Student Participation Form (to record which students participated and which 
did not) and the Assessment Administration Form (to record the timing of the 
assessment and any problems or disturbances which occurred). The teachers 
were able to review the Test Administrator’s Manual before the assessment date 
and raise any questions they had about the procedures with ACER or the State 
and Territory Coordinators responsible for the program. 

The Test Administrator was expected to move around the room while the 
students were working to see that students were following directions and 
answering questions in an appropriate manner. They were allowed to read 
questions to students but could not help the students with the interpretation of 
any of the questions or answer questions about the content of the assessment 
items. 



Quality control 

Quality control was important to the National Assessment Program - Information 
and Communications Technology literacy to minimise systematic error and 
bias. Checks and controls were instituted to ensure that the administration 
within schools was standardised. These included: 

• random sampling of students undertaken by ACER rather than letting 
schools choose their own students; 

• providing detailed manuals; 

• asking the Test Administrator to record student participation on the 
Student Participation Form (a check against the presence or absence of 
data); 

• asking the Test Administrator to complete an Assessment Administration 
Form which recorded the timing of the assessment and any problems or 
disturbances which occurred; and 

• asking the School Contact to verify the information on the Student 
Participation Form and the Assessment Administration Form. 



Summary 

The sampling design and procedures ensured that representative samples of 
Year 6 and Year 10 students were selected. A low level of exclusions and a high 
participation rate ensured that the samples were unbiased. Administrative 
procedures helped to ensure that data of sound quality were collected for 
analysis. 
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Appendix 2 

Sample Characteristics 



This Appendix describes the personai characteristics of the participating 
students at Year 6 and Year 10. At each year ievei, the survey adopted the 
form of a two-stage ciuster sampie design, simiiar to that used by internationai 
assessments such as the OECD Programme for internationai Student 
Assessment (PiSA). 

The sampie was a two-stage (probabiiity proportionai to size) ciuster design 
to ensure that each eiigibie student had an equai chance of being seiected in 
the sampie. identicai procedures were foiiowed for the Year 6 and the Year 10 
samples, in the first stage schools in each stratum were selected, from within 
the strata of State or Territory and sector, with a probability proportional to 
the number of students in the relevant Year level enrolled at that school, in the 
second stage students (other than those students defined as excluded under 
PMRT protocols) were selected at random. This involved obtaining from the 
school a list of all eligible students in the Year level and selecting a random 
sample from the list. Replacement students were selected in case one or more 
of the students declined to participate or is absent on the day of testing. 

The sample design was for a sample of 7,800 students (3,900 at each of Year 6 
and Year 10) from 520 schools (260 at each of Year 6 and Year 10). The achieved 
sample totalled 3,746 Year 6 and 3,647 Year 10 students from 264 primary and 
253 secondary schools across Australia. The participation rates were 96 per 
cent at Year 6 and 93 per cent at Year 10. The survey took place over a two- 
month period from 12 September to 14 November 2005. 

The data presented in the following tables are weighted to allow inferences to 
be made about the Year 6 and Year 10 student populations. Any differences 
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in total numbers of students between tables are due to missing data for those 
variables. 

This Appendix reports on age, gender, Indigenous status, language background 
(country of birth and main language other than English spoken at home), 
socioeconomic background (parental occupation) and geographic location. 
The structure of these variables had been agreed to by the Education Ministers 
as part of the National Assessment Program. The relationships between these 
personal characteristics data reported in this chapter and the cognitive 
achievement data are more fully explored in Chapter 4. 



Age 

MCEETYA protocols mean reporting is against year levels rather than age. 
Nevertheless age differences can account for some of the observed differences 
in performance, and system differences in the distribution of ages in a given year 
level may contribute to observed differences between States and Territories. 
In the achieved sample of participating students, 56 per cent of the Year 10 
students stated they were 15 years old and another 39 per cent said they were 
16 years old or older (Table A2.1). At Year 6, 53 per cent of students were 11 
years old and 42 per cent were 12 years old or older. 

There was some variation in age across the jurisdictions. Compared with 
the Australian average, there were greater proportions of younger students 
in Queensland and Western Australia. By way of contrast, there were larger 
percentages of older students in Tasmania, Victoria, the Australian Capital 
Territory and New South Wales. 



Table A2.1: Age of Students Nationally, by State and Territory and by Year Level 
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Sex 



There were almost equal numbers of males and females in the sample, with 
males comprising 51 per cent of Year 6 students and 52 per cent of Year 10 
students (see Table A2.2). According to Australian Bureau of Statistics data, 
in 2005 males made up 51 per cent of the population at both year levels. From 
Table A2.2 it can be seen that there was an over representation of males in Year 
10 in New South Wales (58 per cent) and of females in Year 6 in the Australian 
Capital Territory (58 per cent). 



Table A2.2: Percentages of Male and Female Students Nationally, by State and Territory and by 
Year Level 





AUST 

% 


NSW 

% 


VIC 

% 


QLD 

% 


SA 

% 


WA 

% 


TAS 

% 


NT 

% 


ACT 

% 


Year 6 


Male 


51.0 


53.8 


50.4 


48.8 


45.5 


52.4 


54.5 


48.3 


41.7 


Female 


49.0 


46.2 


49.6 


51.2 


54.5 


47.6 


45.5 


51.7 


58.3 


Year 10 


Male 


52.3 


58.1 


48.7 


50.5 


49.4 


47.7 


52.3 


53.1 


47.4 


Female 


47.7 


41.9 


51.3 


49.5 


50.6 


52.3 


47.7 


46.9 


52.6 



Geographic Location 

For the purposes of this report, ‘geographic location’ refers to whether a 
student lived in a metropolitan, provincial or remote zone (Jones, 2000). 

• Metropolitan zones included all State and Territory capital cities except 
Darwin and major urban areas with populations above 100,000 (such as 
Geelong, Wollongong and the Gold Coast). 

• Provincial zones took in provincial cities (including Darwin) and provincial 
areas. 

• Remote zones were areas of low accessibility, such as Katherine and 
Coober Pedy. 



Table A2.3: Geographic Location - Percentages of Students Nationally, by State and Territory 
and by Year Level 





AUST 

% 


NSW 

% 


VIC 

% 


QLD 

% 


SA 

% 


WA 

% 


TAS 

% 


NT 

% 


ACT 

% 


Year 6 


Metropolitan 


68.0 


69.5 


70.6 


62.7 


72.9 


71.2 


42.3 


0.0 


98.3 


Provincial 


30.5 


30.4 


29.1 


36.5 


22.0 


23.1 


57.7 


67.9 


1.7 


Remote 


1.5 


0.2 


0.3 


0.8 


5.1 


5.7 


0.0 


32.1 


0.0 


Year 10 


Metropolitan 


71.6 


76.1 


72.4 


69.7 


69.9 


69.1 


38.1 


0.0 


96.1 


Provincial 


26.4 


23.9 


27.6 


26.9 


27.4 


25.3 


60.7 


56.7 


3.9 


Remote 


2.0 


0.0 


0.0 


3.4 


2.6 


5.5 


1.2 


43.3 


0.0 
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Approximately 70 per cent of the students in the National Assessment of ICT 
Literacy lived in metropolitan areas (see Table A2.3). Almost 30 per cent lived 
in provincial areas, while only one to two per cent lived in remote areas. 

There were some variations among the States and Territories in the 
distribution of students across metropolitan, provincial and remote areas. 
Almost all students in the Australian Capital Territory lived in metropolitan 
areas, compared with 42 per cent of Year 6 students and 38 per cent of Year 10 
students in Tasmania and none in the Northern Territory (Darwin is classified 
as a provincial city). The Northern Territory had the greatest number of 
students in remote areas (32 per cent at Year 6 and 43 per cent at Year 10), 
followed by Western Australia (6 per cent at Year 6 and Year 10). 



Indigenous Status 

Six per cent of the Year 6 students and three per cent of the Year 10 students 
sampled identified themselves as being Aboriginal or Torres Strait Islanders 
(see Table A2.4). The highest percentages of Indigenous students were in the 
Northern Territory (21 per cent of Year 6 students and 13 per cent of Year 10 
students). 



Table A2.4: Indigenous Status - Percentages of Students Nationally, by State and Territory and 
by Year Level 





AUST 

% 


NSW 

% 


VIC 

% 


QLD 

% 


SA 

% 


WA 

% 


TAS 

% 


NT 

% 


ACT 

% 


Year 6 


Non-lndigenous 


93.5 


91.4 


95.2 


93.5 


97.8 


94.6 


90.8 


78.6 


98.3 


Indigenous 


6.5 


8.6 


4.8 


6.5 


2.2 


5.4 


9.2 


21.4 


1.7 


Year 10 


Non-lndigenous 


97.0 


97.8 


98.4 


95.0 


97.0 


96.1 


92.9 


87.1 


100.0 


Indigenous 


3.0 


2.2 


1.6 


5.0 


3.0 


3.9 


7.1 


12.9 


0.0 



Indigenous students make up 17 per cent of students from remote locations, 
eight per cent of those from provincial locations and three per cent of those 
from metropolitan locations. 



Language Background 

As shown in Table A2.5 about 25 per cent of sampled students came from 
homes in which languages other than English were spoken (in place of or in 
addition to English). Tasmania had the smallest percentage of students from 
such homes (1 1 per cent of Year 6 students and 6 per cent of Year 10 students), 
while Victoria had the largest percentage (34 per cent of Year 6 students and 
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33 per cent of Year 10 students) followed closely by New South Wales (29 per 
cent of Year 6 students and 32 per cent of Year 10 students). 



Table A2.5: Percentages of Students Speaking a Language Other than English at Home 
Nationally, by State and Territory and by Year Level 





AUST 

% 


NSW 

% 


VIC 

% 


OLD 

% 


SA 

% 


WA 

% 


TAS 

% 


NT 

% 


ACT 

% 


Year 6 


English 


74.3 


71.0 


66.0 


83.6 


78.0 


79.5 


88.8 


75.9 


76.7 


Other than English 


25.7 


29.0 


34.0 


16.4 


22.0 


20.5 


11.2 


24.1 


23.3 


Year 10 


English 


73.7 


68.0 


66.8 


81.9 


82.5 


79.6 


94.1 


75.0 


69.2 


Other than English 


26.3 


32.0 


33.2 


18.1 


17.5 


20.4 


5.9 


25.0 


30.8 



Country of Birth 

Six per cent of the Year 6 students and nine per cent of the Year 10 students 
were not born in Australia or a predominantly English-speaking country (see 
Table A2.6). The percentage of Year 6 students born outside Australia varied 
from two per cent in Tasmania to eight per cent in the Victoria. At Year 10 the 
percentage varied from two per cent in Tasmania to 12 per cent in Victoria. 



Table A2.6: Percentages of Students from Different Countries of Birth Nationally, by State and 
Territory and by Year Level 





AUST 

% 


NSW 

% 


VIC 

% 


OLD 

% 


SA 

% 


WA 

% 


TAS 

% 


NT 

% 


ACT 

% 


Year 6 


Australia / English 


94.5 


94.0 


92.2 


96.2 


97.4 


95.5 


98.0 


96.6 


93.3 


Other 


5.5 


6.0 


7.8 


3.8 


2.6 


4.5 


2.0 


3.4 


6.7 


Year 10 


Australia / English 


91.3 


90.2 


87.8 


94.6 


95.9 


91.8 


97.6 


90.3 


87.2 


Other 


8.7 


9.8 


12.2 


5.4 


4.1 


8.2 


2.4 


9.7 


12.8 



Socioeconomic Background 

information about parental education and parental education was obtained 
to provide an indicator of socioeconomic background, in practice it was 
not possible to use information about parental education because a very 
high percentage of students indicated that they did not know their parents’ 
educational attainment (33 per cent of Year 6 students and 13 per cent 
of Year 10 students). However, only a small percentage of students did not 
provide a response to the question about parental occupation for at least one 
of their parents (six per cent of Year 6 students and six per cent of Year 10 
students). Consequently, parental occupation was used as the only indicator 
of socioeconomic background. This indicator has a stronger tradition as an 
indicator of socioeconomic background than parental education. 
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The parental occupation variable used in this report is a combined variable, 
indicating the higher occupation grouping into which either parent fell. This 
variable is based on questions which asked for both the name of the job the 
student’s mother and father did and a description of what work they did in the 
job. 

The distribution of parental occupations was different for Year 6 and Year 10 
students. Around 25 per cent of Year 6 students, compared to 16 per cent of 
Year 10 students reported that their parents’ highest occupation was in the 
group of unskilled labourers, office, sales and service staff (see Table A2.7). 
Among Year 6 students 29 per cent reported that their parent’s occupation 
was that of a tradesperson or skilled office, sales or service person, 31 per 
cent had parents who were managers or associated professionals and a final 
14 per cent had parents in the senior manager or professionals group. The 
corresponding percentages for Year 10 students were 27 per cent, 39 per cent 
and 18 per cent. 



Table A2.7: Parental Occupation - Percentage of Students Nationally, by State and Territory and 
by Year Level 





AUST 

% 


NSW 

% 


VIC 

% 


OLD 

% 


SA 

% 


WA 

% 


TAS 

% 


NT 

% 


ACT 

% 


Year 6 


Unskilled labourers and 
other staff 


24.5 


24.7 


24.2 


25.0 


23.2 


24.6 


30.5 


22.2 


12.1 


Trades & skilled other 
staff 


28.2 


26.4 


29.1 


30.1 


28.1 


29.2 


22.1 


37.0 


29.3 


Other managers and 
associates 


30.3 


31.0 


30.7 


28.7 


30.7 


28.9 


29.5 


29.6 


36.2 


Senior managers and 
professionals 


13.9 


14.1 


13.5 


13.5 


15.4 


13.5 


13.7 


11.1 


20.7 


Not in paid work for 12 
months 


3.1 


3.8 


2.5 


2.7 


2.6 


3.8 


4.2 


0.0 


1.7 


Year 10 


Unskilled labourers and 
other staff 


15.6 


16.1 


15.2 


14.8 


16.0 


16.5 


22.5 


10.3 


8.0 


Trades & skilled other 
staff 


26.5 


27.0 


21.7 


28.8 


29.8 


27.3 


31.3 


31.0 


22.7 


Other managers and 
associates 


38.4 


36.3 


41.8 


39.8 


37.8 


36.9 


33.8 


37.9 


38.7 


Senior managers and 
professionals 


17.5 


18.5 


20.3 


14.3 


13.7 


16.5 


11.3 


17.2 


30.7 


Not in paid work for 12 
months 


1.9 


2.0 


0.9 


2.3 


2.7 


2.8 


1.3 


3.4 


0.0 



Summary 

The sample of students who completed the national assessment of ICT 
literacy in 2005 was diverse and spanned the range of the Australian school 
populations in Year 6 and Year 10. it was a representative sample in terms of the 
characteristics about which data were gathered. There were some differences 
in these characteristics among States and Territories and some of these 
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characteristics were associated with iCT iiteracy. For that reason it is vaiuabie 
to anaiyse differences among jurisdictions in iCT iiteracy in ways that take 
account of differences in student characteristics as weii as reported overaii 
differences. These anaiyses have been reported in Chapter 4 and Chapter 5. 
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Appendix 3 

Percentage Distributions by 
Proficiency Level 
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Table A3.1: Proportion of Years 6 and 10 Students Achieving at or above Specified Proficiency Levels in ICT Performance, 2005 (per cent) 





NSW 


Vic 


Qld 


WA 


SA 


Tas 


ACT 


NT 


Aust 


% 


Cl 


7o 


Cl 


/o 


Cl 


‘y 

7o 


Cl 


% 


Cl 


7o 


Cl 


/o 


Cl 


7o 


Cl 


% 


Cl 


Year 6 


Level 1 or above 


98.6 


±1.4 


99.8 


±0.5 


97.9 


±1.7 


98.8 


±1.3 


99.5 


±0.8 


98.9 


±1.1 


99.5 


±1.5 


93.4 


±9.4 


98.8 


± 0.5 


Level 2 or above 


89.5 


±3.3 


91.4 


±3.8 


80.7 


±4.8 


82.8 


±4.7 


89.6 


±3.6 


89.7 


±5.1 


91.5 


±4.9 


75.8 


±12.2 


87.4 


± 1.6 


Level 3 or above 


50.5 


±16.6 


57.9 


±6.3 


37.7 


±5.3 


39.6 


±5.4 


51.7 


±5.0 


48.9 


±9.0 


58.4 


±12.5 


36.0 


±10.0 


48.6 


± 3.0 


Level 4 or above 


8.7 


±3.7 


10.5 


±3.5 


4.1 


±1.8 


4.6 


±2.0 


9.0 


±3.5 


8.4 


±4.3 


12.9 


±6.7 


2.8 


±2.6 


7.8 


± 1.6 


Level 5 or above 


0.1 


±0.2 


0.1 


±0.3 


0.1 


±0.3 


0.2 


±0.8 


0.3 


±0.7 


0.4 


±1.1 


0.1 


±0.8 


0.0 


±0.0 


0.1 


± 0.1 


Level 6 or above 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


Year 10 






































Level 1 or above 


100.0 


±0.0 


100.0 


±0.0 


100.0 


±0.0 


100.0 


±0.0 


100.0 


±0.0 


100.0 


±0.0 


100.0 


±0.0 


100.0 


±0.0 


100.00 


± 0.0 


Level 2 or above 


99.7 


±0.4 


99.6 


±0.6 


99.6 


±0.8 


99.8 


±0.5 


99.2 


±0.6 


99.3 


±1.0 


100.0 


±0.0 


99.1 


±2.1 


99.6 


± 0.2 


Level 3 or above 


92.9 


±2.5 


94.1 


±1.9 


94.3 


±2.8 


90.7 


±4.2 


93.9 


±2.4 


91.3 


±4.2 


96.0 


±3.1 


85.6 


±11.3 


93.2 


± 1.2 


Level 4 or above 


61.1 


±7.6 


66.5 


±4.8 


59.5 


±7.4 


55.8 


±6.1 


61.4 


±5.0 


56.4 


±6.4 


65.5 


±11.4 


48.6 


±13.2 


61.2 


± 3.1 


Level 5 or above 


11.7 


±3.5 


17.4 


±4.1 


10.6 


±3.3 


8.2 


±2.9 


12.0 


±3.6 


9.1 


±3.9 


18.0 


±8.9 


7.7 


±5.9 


12.3 


± 1.6 


Level 6 or above 


0.5 


±1.0 


0.7 


±1.2 


0.2 


±0.5 


0.0 


±0.2 


0.4 


±0.6 


0.1 


±0.4 


0.5 


±1.3 


0.0 


±0.0 


0.4 


± 0.4 



a) Minimum standards such as the benchmarks in literacy and numeracy have not been set for ICT literacy performance. Six proficiency levels and a proficient standard are established. 
The proficient standard for ICT literacy performance is set at proficiency level 2 for year 6 and level 3 for year 10 (of levels 1 to 6 or above) a challenging level of performance, with 
students needing to demonstrate more than minimal or elementary skills expected at that year level to be regarded as reaching it. Data represent the proportion of students at or above 
each proficiency level. 

b) The achievement percentages reported in this table include 95 per cent confidence intervals (for example, 80.0 per cent ± 2.7 per cent). 

- Nil or rounded to zero. 





Table A3.2: Proportion of Years 6 and 10 Students Achieving at or above Specified Proficiency Levels in ICT Performance, by Geolocation, Australia, 2005 (per cent) 





Level 1 or above 


Level 2 or above 


Level 3 or above 


Level 4 or above 


Level 5 or above 


Level 6 or above 


°/ 

7o 


Cl 


o/ 

/o 


Cl 


7o 


Cl 


o/ 

7o 


Cl 


7o 


Cl 


°7 

7o 


Cl 


Year 6 


Metropolitan 


99.0 


±0.6 


88.8 


±2.0 


51.9 


±3.8 


9.0 


±2.0 


0.2 


±0.2 


- 


- 


Provincial 


98.5 


±1.2 


85.8 


±3.2 


42.7 


±5.5 


5.3 


±2.4 


0.1 


±0.2 


- 


- 


Remote 


94.2 


±7.7 


73.9 


±15.1 


32.6 


±18.9 


2.5 


±4.2 


0.0 


±0.0 


- 


- 


All locations 


98.8 


±0.5 


87.4 


±1.6 


48.6 


±3.0 


7.8 


±1.6 


0.1 


±0.1 


- 


- 


Year 10 


Metropolitan 


100.0 


±0.0 


99.6 


±0.3 


93.4 


±1.4 


62.8 


±4.1 


13.6 


±2.3 


0.4 


±0.5 


Provincial 


100.0 


±0.0 


99.9 


±0.3 


92.8 


±2.9 


58.6 


±5.7 


10.1 


±3.9 


0.4 


±0.9 


Remote 


100.0 


±0.0 


96.9 


±5.0 


84.6 


±8.0 


45.8 


±9.7 


6.8 


±5.0 


0.1 


±0.6 


All locations 


100.0 


±0.0 


99.6 


±0.2 


93.2 


±1.2 


61.2 


±3.1 


12.3 


±1.6 


0.4 


±0.4 



a) Minimum standards such as the benchmarks in literacy and numeracy have not been set for ICT literacy performance. Six proficiency levels and a proficient standard are established. 
The proficient standard for ICT literacy performance is set at proficiency level 2 for year 6 and level 3 for year 10 (of levels 1 to 6 or above) a challenging level of performance, with 
students needing to demonstrate more than minimal or elementary skills expected at that year level to be regarded as reaching it. Data represent the proportion of students at or above 
each proficiency level. 

b) The achievement percentages reported in this table include 95 per cent confidence intervals (for example, 80.0 per cent ± 2.7 per cent). 

c) Geolocation data are based on the MCEETYA Schools Geographic Location Classification and represent student residential location. 

- Nil or rounded to zero. 






Table A3.3: Proportion of Years 6 and 10 Students Achieving at or above Specified Proficiency Levels in ICT Performance, by Equity Group, Australia, 2005 (per cent) 





Level 1 or above 


Level 2 or above 


Level 3 or above 


Level 4 or above 


Level 5 or above 


Level 6 or above 


Year 6 


Male students 


98.5 


±0.7 


85.6 


±2.6 


45.4 


±4.9 


7.9 


±2.0 


0.2 


±0.3 


- 


- 


Female students 


99.0 


±0.7 


89.3 


±2.0 


52.0 


±4.1 


7.8 


±2.0 


0.1 


±0.2 


- 


- 


Indigenous students 


93.4 


±5.4 


74.8 


±10.6 


29.9 


±12.9 


1.2 


±3.0 


0.1 


±0.4 


- 


- 


LBOTE students 


98.5 


±1.2 


86.5 


±3.7 


48.8 


±6.2 


8.7 


±2.6 


0.0 


±0.0 


- 


- 


All students 


98.8 


±0.5 


87.4 


±1.6 


48.6 


±3.0 


7.8 


±1.6 


0.1 


±0.1 


- 


- 


Year 10 


Male students 


100.0 


±0.0 


99.7 


±0.3 


91.9 


±1.8 


59.6 


±4.2 


11.6 


±2.3 


0.4 


±0.6 


Female students 


100.0 


±0.0 


99.6 


±0.4 


94.8 


±1.7 


62.9 


±3.5 


13.2 


±2.3 


0.4 


±0.5 


Indigenous students 


100.0 


±0.0 


97.3 


±3.9 


79.3 


±10.1 


35.0 


±11.5 


5.8 


±5.8 


0.0 


±0.0 


LBOTE students 


100.0 


±0.0 


99.4 


±0.6 


92.0 


±2.7 


58.6 


±5.6 


12.8 


±3.5 


0.6 


±1.1 


All students 


100.0 


±0.0 


99.6 


±0.2 


93.2 


±1.2 


61.2 


±3.1 


12.3 


±1.6 


0.4 


±0.4 



a) Minimum standards such as the benchmarks in literacy and numeracy have not been set for ICT literacy performance. Six proficiency levels and a proficient standard are established. 
The proficient standard for ICT literacy performance is set at proficiency level 2 for year 6 and level 3 for year 10 (of levels 1 to 6 or above) a challenging level of performance, with 
students needing to demonstrate more than minimal or elementary skills expected at that year level to be regarded as reaching it. Data represent the proportion of students at or above 
each proficiency level. 

b) The achievement percentages reported in this table include 95 per cent confidence intervals (for example, 80.0 per cent ± 2.7 per cent). 

c) Membership of equity groups are based on student responses to the survey questionnaire. 

- Nil or rounded to zero. 
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